date:20140303

[
https://issues.apache.org/jira/browse/CASSANDRA-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sylvain Lebresne resolved CASSANDRA-4988.
-

Resolution: Duplicate
Fix Version/s: (was: 2.0.6)

As said above, there is not real way we can make this happen in a minor
release, and the right solution is basically a sub-part of CASSANDRA-6717, so
marking this as a duplicate of that latter issue.

Fix concurrent addition of collection columns
-

Key: CASSANDRA-4988
URL: https://issues.apache.org/jira/browse/CASSANDRA-4988
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne

It is currently not safe to update the schema by adding multiple collection
columns to the same table. The reason is that with collections, the
comparator embeds a map of names-comparator for each collection columns
(since different maps can have different key type for example). And when
serialized on disk in the schema table, the comparator is serialized as a
string with that map as one column. So if new collection columns are added
concurrently, the addition may not be merged correctly.
One option to fix this would be to stop serializing the names-comparator map
of ColumnToCollectionType in toString(), and do one of:
# reconstruct that map from the information stores in the schema_columns. The
downside I can see is that code-wise this may not be super clean to do.
# change ColumnToCollectionType so that instead of having it's own
names-comparator map, to just store a point to the CFMetaData that contains
it and when it needs to find the exact comparator for a collection column, it
would use CFMetadata.column_metadata directly. The downside is that creating
a dependency from a comparator to a CFMetadata feels a bit backward.
Note sure what's the best solution of the two honestly.
While probably more anecdotal, we also now allow to change the type of the
comparator in some cases (for example updating to BytesType is always
allowed), and doing so concurrently on multiple components of a composite
comparator is also not safe for a similar reason. I'm not sure how to fix
that one.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6694) Slightly More Off-Heap Memtables

2014-03-03 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917897#comment-13917897
 ] 

Benedict commented on CASSANDRA-6694:
-

Pushed another update, with a slight modification that improves the behaviour 
when deleting stale entries in KeysSearcher and CompositesSearcher 

 Slightly More Off-Heap Memtables
 

 Key: CASSANDRA-6694
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6694
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 2.1 beta2


 The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as 
 the on-heap overhead is still very large. It should not be tremendously 
 difficult to extend these changes so that we allocate entire Cells off-heap, 
 instead of multiple BBs per Cell (with all their associated overhead).
 The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 
 bytes per cell on average for the btree overhead, for a total overhead of 
 around 20-22 bytes). This translates to 8-byte object overhead, 4-byte 
 address (we will do alignment tricks like the VM to allow us to address a 
 reasonably large memory space, although this trick is unlikely to last us 
 forever, at which point we will have to bite the bullet and accept a 24-byte 
 per cell overhead), and 4-byte object reference for maintaining our internal 
 list of allocations, which is unfortunately necessary since we cannot safely 
 (and cheaply) walk the object graph we allocate otherwise, which is necessary 
 for (allocation-) compaction and pointer rewriting.
 The ugliest thing here is going to be implementing the various CellName 
 instances so that they may be backed by native memory OR heap memory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6624) General protection fault


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-6624:


Priority: Major  (was: Critical)

 General protection fault
 

 Key: CASSANDRA-6624
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6624
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Linux s41083 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 
 17:37:58 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
 java version 1.7.0_51
 Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
 Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
Reporter: Mateusz Gajewski
 Fix For: 2.0.6

 Attachments: system.log


 Hi,
 Yesterday I got General Protection Fault in cassandra 2.0.3 process while 
 stress testing it.
 Jan 26 23:19:43 s41083 kernel: [461545.017756] java[192074] general 
 protection ip:7fea558c6ae7 sp:7fe959844bf0 error:0 in 
 libc-2.15.so[7fea5588d000+1b5000]
 It just died while doing compactation and restarted couple of times several 
 minutes after that.
 System log attached



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6624) General protection fault


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917911#comment-13917911
 ] 

Sylvain Lebresne commented on CASSANDRA-6624:
-

Stating the obvious, but while this could probably be triggered by some of our 
JNA/Unsafe usages, a GPF also potentially have a JVM bug smell. But in any 
case, without at least a core dump or a way to reproduce, I'm not sure there is 
much we'll be able to do here.

 General protection fault
 

 Key: CASSANDRA-6624
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6624
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Linux s41083 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 
 17:37:58 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
 java version 1.7.0_51
 Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
 Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
Reporter: Mateusz Gajewski
 Fix For: 2.0.6

 Attachments: system.log


 Hi,
 Yesterday I got General Protection Fault in cassandra 2.0.3 process while 
 stress testing it.
 Jan 26 23:19:43 s41083 kernel: [461545.017756] java[192074] general 
 protection ip:7fea558c6ae7 sp:7fe959844bf0 error:0 in 
 libc-2.15.so[7fea5588d000+1b5000]
 It just died while doing compactation and restarted couple of times several 
 minutes after that.
 System log attached



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6206) Thrift socket listen backlog


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917924#comment-13917924
 ] 

Sylvain Lebresne commented on CASSANDRA-6206:
-

What part of this exactly requires THRIFT-1868? Is the Committed to trunk 
means that this is fixed in 2.1 but cannot be fixed in 2.0 without THRIFT-1868? 
Cause that thrift ticket has had a patch waiting for review since last 
September. I've pinged them on the issue, but if there is no movement soon and 
that we're good in 2.1, maybe we should just say that it's 'fix for' 2.1 and 
move on.

 Thrift socket listen backlog
 

 Key: CASSANDRA-6206
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6206
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Debian Linux, Java 7
Reporter: Nenad Merdanovic
 Fix For: 2.0.6

 Attachments: cassandra-v2.patch, cassandra.patch


 Although Thrift is a depreciated method of accessing Cassandra, default 
 backlog is way too low on that socket. It shouldn't be a problem to implement 
 it and I am including a POC patch for this (sorry, really low on time with 
 limited Java knowledge so just to give an idea).
 This is an old report which was never addressed and the bug remains till this 
 day, except in my case I have a much larger scale application with 3rd party 
 software which I cannot modify to include connection pooling:
 https://issues.apache.org/jira/browse/CASSANDRA-1663
 There is also a pending change in the Thrift itself which Cassandra should be 
 able to use for parts using TServerSocket (SSL):
 https://issues.apache.org/jira/browse/THRIFT-1868



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6624) General protection fault

2014-03-03 Thread Mateusz Gajewski (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917935#comment-13917935
 ] 

Mateusz Gajewski commented on CASSANDRA-6624:
-

Hi there,

After upgrade to 2.0.6 and some changes to kernel and firmware upgrades I have 
not encountered this behaviour. I think that this bug report can be closed for 
now as it's origin is not clear enough.

 General protection fault
 

 Key: CASSANDRA-6624
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6624
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Linux s41083 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 
 17:37:58 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
 java version 1.7.0_51
 Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
 Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
Reporter: Mateusz Gajewski
 Fix For: 2.0.6

 Attachments: system.log


 Hi,
 Yesterday I got General Protection Fault in cassandra 2.0.3 process while 
 stress testing it.
 Jan 26 23:19:43 s41083 kernel: [461545.017756] java[192074] general 
 protection ip:7fea558c6ae7 sp:7fe959844bf0 error:0 in 
 libc-2.15.so[7fea5588d000+1b5000]
 It just died while doing compactation and restarted couple of times several 
 minutes after that.
 System log attached



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (CASSANDRA-6624) General protection fault


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-6624.
-

Resolution: Cannot Reproduce

Alright, let's close as Cannot Reproduce for now. If someone run into it 
again and has more info to provide, we can reopen.

 General protection fault
 

 Key: CASSANDRA-6624
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6624
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Linux s41083 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 
 17:37:58 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
 java version 1.7.0_51
 Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
 Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
Reporter: Mateusz Gajewski
 Fix For: 2.0.6

 Attachments: system.log


 Hi,
 Yesterday I got General Protection Fault in cassandra 2.0.3 process while 
 stress testing it.
 Jan 26 23:19:43 s41083 kernel: [461545.017756] java[192074] general 
 protection ip:7fea558c6ae7 sp:7fe959844bf0 error:0 in 
 libc-2.15.so[7fea5588d000+1b5000]
 It just died while doing compactation and restarted couple of times several 
 minutes after that.
 System log attached



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6624) General protection fault

2014-03-03 Thread Mateusz Gajewski (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917946#comment-13917946
 ] 

Mateusz Gajewski commented on CASSANDRA-6624:
-

Fine, next time I will submit core dumps also :)

 General protection fault
 

 Key: CASSANDRA-6624
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6624
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Linux s41083 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 
 17:37:58 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
 java version 1.7.0_51
 Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
 Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
Reporter: Mateusz Gajewski
 Fix For: 2.0.6

 Attachments: system.log


 Hi,
 Yesterday I got General Protection Fault in cassandra 2.0.3 process while 
 stress testing it.
 Jan 26 23:19:43 s41083 kernel: [461545.017756] java[192074] general 
 protection ip:7fea558c6ae7 sp:7fe959844bf0 error:0 in 
 libc-2.15.so[7fea5588d000+1b5000]
 It just died while doing compactation and restarted couple of times several 
 minutes after that.
 System log attached



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6285) LCS compaction failing with Exception


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917950#comment-13917950
 ] 

Viktor Kuzmin commented on CASSANDRA-6285:
--

https://www.dropbox.com/s/3ldg10zh7qvva27/cassandra-attack.jar

Schema is located inside jar file - cassandra.txt

1. Start cassandra
2. java -jar cassandra-attack.jar
3. Stop cassandra
4. Start cassandra - commit logs will be corrupted.

 LCS compaction failing with Exception
 -

 Key: CASSANDRA-6285
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6285
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2
Reporter: David Sauer
Assignee: Marcus Eriksson
 Fix For: 2.0.6

 Attachments: compaction_test.py


 After altering everything to LCS the table OpsCenter.rollups60 amd one other 
 none OpsCenter-Table got stuck with everything hanging around in L0.
 The compaction started and ran until the logs showed this:
 ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java 
 (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime]
 java.lang.RuntimeException: Last written key 
 DecoratedKey(1326283851463420237, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564)
  = current key DecoratedKey(954210699457429663, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f)
  writing into 
 /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
   at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:724)
 Moving back to STC worked to keep the compactions running.
 Especialy my own Table i would like to move to LCS.
 After a major compaction with STC the move to LCS fails with the same 
 Exception.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6526) CQLSSTableWriter addRow(MapString, Object values) does not work as documented.


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-6526:


Attachment: 6526.txt

Attaching simple patch to fix the bug mentioned and add some doc on when you 
should lowercase column names.

 CQLSSTableWriter addRow(MapString, Object values) does not work as 
 documented.
 

 Key: CASSANDRA-6526
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6526
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Yariv Amar
 Fix For: 2.0.6

 Attachments: 6526.txt

   Original Estimate: 24h
  Remaining Estimate: 24h

 There are 2 bugs in the method
 {code}
 addRow(MapString, Object values)
 {code}
 First issue is that the map bmust/b contain all the column names as keys 
 in the map otherwise the addRow fails (with InvalidRequestException Invalid 
 number of arguments, expecting %d values but got %d).
 Second Issue is that the keys in the map must be in lower-case otherwise they 
 may not be found in the map, which will result in a NPE during decompose.
 h6. SUGGESTED SOLUTION:
 Fix the addRow method with:
 {code}
 public CQLSSTableWriter addRow(MapString, Object values)
 throws InvalidRequestException, IOException
 {
 int size = boundNames.size();
 MapString, ByteBuffer rawValues = new HashMap(size);
 for (int i = 0; i  size; i++) {
 ColumnSpecification spec = boundNames.get(i);
 String colName = spec.name.toString();
 rawValues.put(colName, values.get(colName) == null ? null : 
 ((AbstractType)spec.type).decompose(values.get(colName)));
 }
 return rawAddRow(rawValues);
 }
 {code}
 When creating the new Map for the insert we need to go over all columns and 
 apply null to missing columns.
 Fix the method documentation add this line:
 {code}
  * p
  * Keys in the map bmust/b be in lower case, otherwise their value 
 will be null.
  *
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6733) Upgrade of 1.2.11 to 2.0.5 make IllegalArgumentException in Buffer.limit on read of a super column family


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917961#comment-13917961
 ] 

Sylvain Lebresne commented on CASSANDRA-6733:
-

[~vijay2...@yahoo.com] marking you as reviewer since you were reviewer on 
CASSANDRA-3237 which this is basically a left-over. Feel free to complain if 
you don't have time to look at it (but it's not really a big patch).

 Upgrade of 1.2.11 to 2.0.5 make IllegalArgumentException in Buffer.limit on 
 read of a super column family
 -

 Key: CASSANDRA-6733
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6733
 Project: Cassandra
  Issue Type: Bug
Reporter: Nicolas Lalevée
Assignee: Sylvain Lebresne
 Fix For: 2.0.6

 Attachments: 6733.txt, QaUser_user_view_node1.tgz, 
 QaUser_user_view_node2.tgz


 We have a super column family which was first created with a 1.0.x. Then 
 upgraded to 1.1.x, then to 1.2.11, and now to 2.0.5.
 {noformat}
 cqlsh:QaUser desc table user_view;
 CREATE TABLE user_view (
   key bigint,
   column1 varint,
   column2 text,
   value counter,
   PRIMARY KEY (key, column1, column2)
 ) WITH COMPACT STORAGE AND
   bloom_filter_fp_chance=0.01 AND
   caching='KEYS_ONLY' AND
   comment='' AND
   dclocal_read_repair_chance=0.00 AND
   gc_grace_seconds=864000 AND
   index_interval=128 AND
   read_repair_chance=1.00 AND
   replicate_on_write='true' AND
   populate_io_cache_on_flush='false' AND
   default_time_to_live=0 AND
   speculative_retry='99.0PERCENTILE' AND
   memtable_flush_period_in_ms=0 AND
   compaction={'class': 'SizeTieredCompactionStrategy'} AND
   compression={'sstable_compression': 'SnappyCompressor'};
 {noformat}
 With cqlsh, the following query was doing a timeout:
 {noformat}
 select * from user_view where key = 3 and column1 = 1 and column2 = 
 '20130218';
 {noformat}
 In the log of cassandra, we could read:
 {noformat}
 ERROR [ReadStage:1385] 2014-02-19 14:45:19,549 CassandraDaemon.java (line 
 192) Exception in thread Thread[ReadStage:1385,5,main]
 java.lang.IllegalArgumentException
 at java.nio.Buffer.limit(Buffer.java:267)
 at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
 at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
 at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:82)
 at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:35)
 at 
 org.apache.cassandra.db.marshal.AbstractType$1.compare(AbstractType.java:63)
 at 
 org.apache.cassandra.db.marshal.AbstractType$1.compare(AbstractType.java:60)
 at java.util.Collections.indexedBinarySearch(Collections.java:377)
 at java.util.Collections.binarySearch(Collections.java:365)
 at 
 org.apache.cassandra.io.sstable.IndexHelper.indexFor(IndexHelper.java:144)
 at 
 org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlockFetcher.setNextSlice(IndexedSliceReader.java:262)
 at 
 org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlockFetcher.init(IndexedSliceReader.java:255)
 at 
 org.apache.cassandra.db.columniterator.IndexedSliceReader.init(IndexedSliceReader.java:91)
 at 
 org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:65)
 at 
 org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:42)
 at 
 org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:167)
 at 
 org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
 at 
 org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:250)
 at 
 org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1560)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1379)
 at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
 at 
 org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
 at 
 org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

[jira] [Updated] (CASSANDRA-6778) FBUtilities.singleton() should use the CF comparator


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-6778:


Reviewer: Tyler Hobbs

 FBUtilities.singleton() should use the CF comparator
 

 Key: CASSANDRA-6778
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6778
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 2.0.6

 Attachments: 0001-Proper-comparison-for-singleton-sorted-set.txt, 
 0002-Use-comparator-instead-of-BB.equals.txt


 We sometimes use FBUtilities.singleton() to created a SortedSet for 
 NamesQueryFilter. However, the set created by that method does not use the CF 
 comparator, so that it use ByteBuffer comparison/equality for methods like 
 contains(). And this might not be ok if it turns that the comparator is so 
 that 2 column name can be equal without their binary representation being 
 equal, and as it turns out at least IntegerType, DecimalType (because they 
 let you put arbitrary many zeros in front of the binary encoding) have such 
 property (BooleanType should also have that property though it doesn't in 
 practice which I think that's a bug, but that's for another ticket).
 I'll note that CASSANDRA-6733 contains an example where this matter.  
 However, in practice, only SELECT on compact tables that select just one 
 column can ever ran into that and you'd only run into it if your client 
 insert useless zeros in its IntegerType/DecimalType binary representation, 
 which ought to be not common in the first place. It's still wrong and should 
 be fixed.
 Patch attached to include the comparator in FBUtilities.singleton. I also 
 found 2 other small places where we were using ByteBuffer.equals() where the 
 comparator should be used instead and attaching a 2nd patch for those.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6733) Upgrade of 1.2.11 to 2.0.5 make IllegalArgumentException in Buffer.limit on read of a super column family


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-6733:


Reviewer: Vijay

 Upgrade of 1.2.11 to 2.0.5 make IllegalArgumentException in Buffer.limit on 
 read of a super column family
 -

 Key: CASSANDRA-6733
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6733
 Project: Cassandra
  Issue Type: Bug
Reporter: Nicolas Lalevée
Assignee: Sylvain Lebresne
 Fix For: 2.0.6

 Attachments: 6733.txt, QaUser_user_view_node1.tgz, 
 QaUser_user_view_node2.tgz


 We have a super column family which was first created with a 1.0.x. Then 
 upgraded to 1.1.x, then to 1.2.11, and now to 2.0.5.
 {noformat}
 cqlsh:QaUser desc table user_view;
 CREATE TABLE user_view (
   key bigint,
   column1 varint,
   column2 text,
   value counter,
   PRIMARY KEY (key, column1, column2)
 ) WITH COMPACT STORAGE AND
   bloom_filter_fp_chance=0.01 AND
   caching='KEYS_ONLY' AND
   comment='' AND
   dclocal_read_repair_chance=0.00 AND
   gc_grace_seconds=864000 AND
   index_interval=128 AND
   read_repair_chance=1.00 AND
   replicate_on_write='true' AND
   populate_io_cache_on_flush='false' AND
   default_time_to_live=0 AND
   speculative_retry='99.0PERCENTILE' AND
   memtable_flush_period_in_ms=0 AND
   compaction={'class': 'SizeTieredCompactionStrategy'} AND
   compression={'sstable_compression': 'SnappyCompressor'};
 {noformat}
 With cqlsh, the following query was doing a timeout:
 {noformat}
 select * from user_view where key = 3 and column1 = 1 and column2 = 
 '20130218';
 {noformat}
 In the log of cassandra, we could read:
 {noformat}
 ERROR [ReadStage:1385] 2014-02-19 14:45:19,549 CassandraDaemon.java (line 
 192) Exception in thread Thread[ReadStage:1385,5,main]
 java.lang.IllegalArgumentException
 at java.nio.Buffer.limit(Buffer.java:267)
 at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
 at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
 at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:82)
 at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:35)
 at 
 org.apache.cassandra.db.marshal.AbstractType$1.compare(AbstractType.java:63)
 at 
 org.apache.cassandra.db.marshal.AbstractType$1.compare(AbstractType.java:60)
 at java.util.Collections.indexedBinarySearch(Collections.java:377)
 at java.util.Collections.binarySearch(Collections.java:365)
 at 
 org.apache.cassandra.io.sstable.IndexHelper.indexFor(IndexHelper.java:144)
 at 
 org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlockFetcher.setNextSlice(IndexedSliceReader.java:262)
 at 
 org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlockFetcher.init(IndexedSliceReader.java:255)
 at 
 org.apache.cassandra.db.columniterator.IndexedSliceReader.init(IndexedSliceReader.java:91)
 at 
 org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:65)
 at 
 org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:42)
 at 
 org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:167)
 at 
 org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
 at 
 org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:250)
 at 
 org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1560)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1379)
 at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
 at 
 org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
 at 
 org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:724)
 {noformat}
 I tried launching repair on our 2 nodes, nothing improved.
 I tried launching a major compaction on this column family, the query doesn't 
 fail anymore and return expected results;
 This happens

[jira] [Updated] (CASSANDRA-6782) setting TTL on some columns seems to expire whole row


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-6782:


Reviewer: Aleksey Yeschenko

 setting TTL on some columns seems to expire whole row
 -

 Key: CASSANDRA-6782
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6782
 Project: Cassandra
  Issue Type: Bug
 Environment: java version 1.7.0_51
 cassandra from trunk: 9c4824d6c476
Reporter: Russ Hatch
Assignee: Sylvain Lebresne
 Fix For: 2.0.6

 Attachments: 6782.txt


 I create a table with 4 columns, set a ttl on 2 of the columns and when the 
 TTL is up, the entire row disappears.
 {noformat}
 cqlsh:myks CREATE TABLE paging_test (
 ...   id int,
 ...   mytext text,
 ...   anothervalue text,
 ...   somevalue text,
 ...   PRIMARY KEY (id, mytext)
 ... );
 cqlsh:myks insert into paging_test (id, mytext, anothervalue, somevalue) 
 values (1, 'foo', 'some', 'another');
 cqlsh:myks select * from paging_test;
  id | mytext | anothervalue | somevalue
 ++--+---
   1 |foo | some |   another
 (1 rows)
 cqlsh:myks update paging_test using ttl 10
 ...   set somevalue='one', anothervalue='two'
 ...   where id = 1 and mytext = 'foo';
 cqlsh:myks select * from paging_test;
  id | mytext | anothervalue | somevalue
 ++--+---
   1 |foo |  two |   one
 (1 rows)
 cqlsh:myks -- wait for it
 cqlsh:myks select * from paging_test;
 (0 rows)
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-4501) Gossip the ip and port used by the native protocol


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-4501:


Fix Version/s: (was: 2.0.6)
   2.1

 Gossip the ip and port used by the native protocol
 --

 Key: CASSANDRA-4501
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4501
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Brandon Williams
Priority: Minor
 Fix For: 2.1


 Same as we gossip the rpc_address, we should add gossipping of the native 
 transport address (including the port). CASSANDRA-4480 has one reason why we 
 would want to do that.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-5708) Add DELETE ... IF EXISTS to CQL3


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-5708:


Assignee: Tyler Hobbs

 Add DELETE ... IF EXISTS to CQL3
 

 Key: CASSANDRA-5708
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5708
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.0.6


 I've been slightly lazy in CASSANDRA-5443 and didn't added a {{DELETE .. IF 
 EXISTS}} syntax to CQL because it wasn't immediately clear what was the 
 correct condition to use for the IF EXISTS. But at least for CQL3 tables, 
 this is in fact pretty easy to do using the row marker so we should probably 
 add it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-4501) Gossip the ip and port used by the native protocol


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917972#comment-13917972
 ] 

Sylvain Lebresne commented on CASSANDRA-4501:
-

How big of a deal is that to add a new native_transport_address to gossip? It's 
not so much that I care about allowing to set thrift and the native transport 
on different address, but it kind of bugs me that the native protocol reuse 
rpc_address somewhat implicitly without having it's own setting, it's not 
extremely consistent in the yaml and it makes it feels like the native 
transport is somewhat of a 2nd class citizen compared to the thrift one. Not a 
huge deal obviously, but figured that if we want to clean that up a bit for the 
future, we should as well do it now before/simultaneously with CASSANDRA-5899 
so it's coherent.

 Gossip the ip and port used by the native protocol
 --

 Key: CASSANDRA-4501
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4501
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Brandon Williams
Priority: Minor
 Fix For: 2.1


 Same as we gossip the rpc_address, we should add gossipping of the native 
 transport address (including the port). CASSANDRA-4480 has one reason why we 
 would want to do that.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-5708) Add DELETE ... IF EXISTS to CQL3


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917973#comment-13917973
 ] 

Sylvain Lebresne commented on CASSANDRA-5708:
-

bq.  We just probably require a flag in SP.cas() to say apply the cas if the 
result fetch is not null.

I'll note that now that we've abstracted conditions in SP.cas() this should be 
simpler/cleaner to do.

 Add DELETE ... IF EXISTS to CQL3
 

 Key: CASSANDRA-5708
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5708
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.0.6


 I've been slightly lazy in CASSANDRA-5443 and didn't added a {{DELETE .. IF 
 EXISTS}} syntax to CQL because it wasn't immediately clear what was the 
 correct condition to use for the IF EXISTS. But at least for CQL3 tables, 
 this is in fact pretty easy to do using the row marker so we should probably 
 add it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6766) allow now() - uuid compatibility


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917979#comment-13917979
 ] 

Sylvain Lebresne commented on CASSANDRA-6766:
-

I'll remark that because the code for functions is relatively generic, we can't 
just special this for now() (nor should we really). Meaning that what we need 
here is to introduce a bit of subtyping in the type system and make timeuuid a 
subtype of uuid. Which in itself if probably not that hard, but mentioning it 
because it's still probably a bit more involved than what the ticket title 
imply.

 allow now() - uuid compatibility
 -

 Key: CASSANDRA-6766
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6766
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 2.0.6


 Bad Request: Type error: cannot assign result of function now (type timeuuid) 
 to id (type uuid)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Issue Comment Deleted] (CASSANDRA-6772) Cassandra inter data center communication broken

2014-03-03 Thread Ananthkumar K S (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ananthkumar K S updated CASSANDRA-6772:
---

Comment: was deleted

(was: Any updates on this?)

 Cassandra inter data center communication broken
 

 Key: CASSANDRA-6772
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6772
 Project: Cassandra
  Issue Type: Bug
 Environment: CentOS 6.0
Reporter: Ananthkumar K S
Priority: Blocker

 I have two data enters DC1 and DC2. Both communicate via a private link. 
 Yesterday, we had a problem with a private link for 10 mins. From the time 
 the problem was resolved, nodes in both data centers are not able to 
 communicate with each other. When I do a nodetool status on a node in DC1, 
 the nodes in DC2 are stated as down. When tried in DC2, nodes in DC1 are 
 shown as down .
 But in the cassandra logs, we can clearly see that handshaking is failing 
 every 5 seconds for communication between data centres. At TCP level, there 
 are too many fin_wait1 generated by cassandra which is still a puzzle . 
 Closed_wait top transitions due to this is very high. Due to this kind of 
 problem of TCP listen drops, we moved from 2.0.1 to 2.0.3. In 2.0.1, it was 
 within data center itself. But here it's between data centers. If it has 
 anything to do with the snitch configuration, I am using 
 GossipingPropertyFileSnitch.
 This clearly started happening post private link failure. Any idea on this?
 Cassandra version used is 2.0.3



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6772) Cassandra inter data center communication broken

2014-03-03 Thread Ananthkumar K S (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917993#comment-13917993
 ] 

Ananthkumar K S commented on CASSANDRA-6772:


Adding to the details mentioned above, we had another outage at the server 
level in the datacenter but not at the ISP level. But this time 
around,cassandra did its part well by getting back its connections once the 
server was up. So , an ISP level private link failure mainly causes these kind 
of problems. Guess that give you some more clarity into the issue.

 Cassandra inter data center communication broken
 

 Key: CASSANDRA-6772
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6772
 Project: Cassandra
  Issue Type: Bug
 Environment: CentOS 6.0
Reporter: Ananthkumar K S
Priority: Blocker

 I have two data enters DC1 and DC2. Both communicate via a private link. 
 Yesterday, we had a problem with a private link for 10 mins. From the time 
 the problem was resolved, nodes in both data centers are not able to 
 communicate with each other. When I do a nodetool status on a node in DC1, 
 the nodes in DC2 are stated as down. When tried in DC2, nodes in DC1 are 
 shown as down .
 But in the cassandra logs, we can clearly see that handshaking is failing 
 every 5 seconds for communication between data centres. At TCP level, there 
 are too many fin_wait1 generated by cassandra which is still a puzzle . 
 Closed_wait top transitions due to this is very high. Due to this kind of 
 problem of TCP listen drops, we moved from 2.0.1 to 2.0.3. In 2.0.1, it was 
 within data center itself. But here it's between data centers. If it has 
 anything to do with the snitch configuration, I am using 
 GossipingPropertyFileSnitch.
 This clearly started happening post private link failure. Any idea on this?
 Cassandra version used is 2.0.3



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6790) Triggers are broken in trunk because of imutable list


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-6790:


Fix Version/s: (was: 2.1 beta1)
   2.1 beta2

 Triggers are broken in trunk because of imutable list
 -

 Key: CASSANDRA-6790
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6790
 Project: Cassandra
  Issue Type: Bug
Reporter: Edward Capriolo
Assignee: Edward Capriolo
 Fix For: 2.1 beta2


 The trigger code is uncovered by any tests (that I can find). When inserting 
 single columns an immutable list is created. When the trigger attempts to 
 edit this list the operation fails.
 Fix coming shortly.
 {noformat}
 java.lang.UnsupportedOperationException
 at java.util.AbstractList.add(AbstractList.java:148)
 at java.util.AbstractList.add(AbstractList.java:108)
 at 
 java.util.AbstractCollection.addAll(AbstractCollection.java:342)
 at 
 org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:522)
 at 
 org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1084)
 at 
 org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1066)
 at 
 org.apache.cassandra.thrift.CassandraServer.internal_insert(CassandraServer.java:676)
 at 
 org.apache.cassandra.thrift.CassandraServer.insert(CassandraServer.java:697)
 at 
 org.apache.cassandra.triggers.TriggerTest.createATriggerWithCqlAndReadItBackFromthrift(TriggerTest.java:108)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
 at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
 at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
 at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
 at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
 at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
 at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
 at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
 at 
 org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
 at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
 at 
 org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
 at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
 at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
 at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
 at 
 org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
 at 
 org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
 at 
 org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
 at 
 org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
 at 
 org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
 at 
 org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6632) Rename row cache to partition cache


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918038#comment-13918038
 ] 

Sylvain Lebresne commented on CASSANDRA-6632:
-

As I mentioned in CASSANDRA-5357, it occurred to me that the row cache actually 
does cache CQL3 rows, even though the meaning of row is different that the 
original intent, so maybe it's not worth bothering going into a rename. In 
fact, now that we don't cache fully partition necessarily, one could make the 
argument that row_cache is kind of better than partition_cache.

 Rename row cache to partition cache
 ---

 Key: CASSANDRA-6632
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6632
 Project: Cassandra
  Issue Type: Improvement
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson
Priority: Minor
 Fix For: 2.1 beta2


 We should rename row cache into partition cache to avoid confusion.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-6794) Optimise slab allocator to enable higher number of column families

2014-03-03 Thread Jeremy Hanna (JIRA)

Jeremy Hanna created CASSANDRA-6794:
---

 Summary: Optimise slab allocator to enable higher number of column 
families
 Key: CASSANDRA-6794
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6794
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jeremy Hanna
Assignee: Benedict


Currently the slab allocator allocates 1MB per column family.  This has been 
very beneficial for gc efficiency.  However, it makes it more difficult to have 
large numbers of column families.

It would be preferable to have a more intelligent way to allocate slabs so that 
there is more flexibility between slab allocator and non-slab allocator 
behaviour.

A simple first step is to ramp up size of slabs from small (say  8KB) when 
empty, to 1MB after a few slabs.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Fix resetAndTruncate:ing CompressionMetadata

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 b3a9a4434 - 41d8a5f48


Fix resetAndTruncate:ing CompressionMetadata

Patch by kvaster, reviewed by marcuse for CASSANDRA-6791


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/41d8a5f4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/41d8a5f4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/41d8a5f4

Branch: refs/heads/cassandra-2.0
Commit: 41d8a5f4861c37ff8e22344418e9277236672c1f
Parents: b3a9a44
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 11:35:03 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 14:21:00 2014 +0100

--
 CHANGES.txt |  1 +
 .../io/compress/CompressedSequentialWriter.java |  2 +-
 .../CompressedRandomAccessReaderTest.java   | 42 
 3 files changed, 44 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/41d8a5f4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3e73f91..6de11c5 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -28,6 +28,7 @@
  * Disallow post-query re-ordering when paging (CASSANDRA-6722)
  * Fix potential paging bug with deleted columns (CASSANDRA-6748)
  * Fix NPE on BulkLoader caused by losing StreamEvent (CASSANDRA-6636)
+ * Fix truncating compression metadata (CASSANDRA-6791)
 Merged from 1.2:
  * Add CMSClassUnloadingEnabled JVM option (CASSANDRA-6541)
  * Catch memtable flush exceptions during shutdown (CASSANDRA-6735)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/41d8a5f4/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java 
b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
index 54b990f..eef5b17 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
@@ -231,7 +231,7 @@ public class CompressedSequentialWriter extends 
SequentialWriter
 
 // truncate data and index file
 truncate(chunkOffset);
-metadataWriter.resetAndTruncate(realMark.nextChunkIndex);
+metadataWriter.resetAndTruncate(realMark.nextChunkIndex - 1);
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/41d8a5f4/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index ee32a0e..3c9dfe5 100644
--- 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++ 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@ -19,11 +19,13 @@
 package org.apache.cassandra.io.compress;
 
 import java.io.*;
+import java.util.Collections;
 import java.util.Random;
 
 import org.junit.Test;
 
 import org.apache.cassandra.db.marshal.BytesType;
+import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.apache.cassandra.io.sstable.SSTableMetadata;
 import org.apache.cassandra.io.util.*;
@@ -48,6 +50,46 @@ public class CompressedRandomAccessReaderTest
 testResetAndTruncate(File.createTempFile(compressed, 1), true, 10);
 testResetAndTruncate(File.createTempFile(compressed, 2), true, 
CompressionParameters.DEFAULT_CHUNK_LENGTH);
 }
+@Test
+public void test6791() throws IOException, ConfigurationException
+{
+File f = File.createTempFile(compressed6791_, 3);
+String filename = f.getAbsolutePath();
+try
+{
+
+SSTableMetadata.Collector sstableMetadataCollector = 
SSTableMetadata.createCollector(BytesType.instance).replayPosition(null);
+CompressedSequentialWriter writer = new 
CompressedSequentialWriter(f, filename + .metadata, false, new 
CompressionParameters(SnappyCompressor.instance, 32, Collections.String, 
StringemptyMap()), sstableMetadataCollector);
+
+for (int i = 0; i  20; i++)
+writer.write(x.getBytes());
+
+FileMark mark = writer.mark();
+// write enough garbage to create new chunks:
+for (int i = 0; i  40; ++i)
+writer.write(y.getBytes());
+
+writer.resetAndTruncate(mark);
+
+for (int i = 0; i  20; i++)
+

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e80564f3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e80564f3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e80564f3

Branch: refs/heads/trunk
Commit: e80564f398c6278f6a6a28943e06ec800d5e68f7
Parents: 7d8092b e4437bc
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 14:27:15 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 14:27:15 2014 +0100

--
 CHANGES.txt |  1 +
 .../io/compress/CompressedSequentialWriter.java |  2 +-
 .../CompressedRandomAccessReaderTest.java   | 42 
 3 files changed, 44 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e80564f3/CHANGES.txt
--

[1/3] git commit: Fix resetAndTruncate:ing CompressionMetadata

Repository: cassandra
Updated Branches:
  refs/heads/trunk 7d8092b66 - e80564f39


Fix resetAndTruncate:ing CompressionMetadata

Patch by kvaster, reviewed by marcuse for CASSANDRA-6791


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/41d8a5f4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/41d8a5f4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/41d8a5f4

Branch: refs/heads/trunk
Commit: 41d8a5f4861c37ff8e22344418e9277236672c1f
Parents: b3a9a44
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 11:35:03 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 14:21:00 2014 +0100

--
 CHANGES.txt |  1 +
 .../io/compress/CompressedSequentialWriter.java |  2 +-
 .../CompressedRandomAccessReaderTest.java   | 42 
 3 files changed, 44 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/41d8a5f4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3e73f91..6de11c5 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -28,6 +28,7 @@
  * Disallow post-query re-ordering when paging (CASSANDRA-6722)
  * Fix potential paging bug with deleted columns (CASSANDRA-6748)
  * Fix NPE on BulkLoader caused by losing StreamEvent (CASSANDRA-6636)
+ * Fix truncating compression metadata (CASSANDRA-6791)
 Merged from 1.2:
  * Add CMSClassUnloadingEnabled JVM option (CASSANDRA-6541)
  * Catch memtable flush exceptions during shutdown (CASSANDRA-6735)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/41d8a5f4/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java 
b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
index 54b990f..eef5b17 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
@@ -231,7 +231,7 @@ public class CompressedSequentialWriter extends 
SequentialWriter
 
 // truncate data and index file
 truncate(chunkOffset);
-metadataWriter.resetAndTruncate(realMark.nextChunkIndex);
+metadataWriter.resetAndTruncate(realMark.nextChunkIndex - 1);
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/41d8a5f4/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index ee32a0e..3c9dfe5 100644
--- 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++ 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@ -19,11 +19,13 @@
 package org.apache.cassandra.io.compress;
 
 import java.io.*;
+import java.util.Collections;
 import java.util.Random;
 
 import org.junit.Test;
 
 import org.apache.cassandra.db.marshal.BytesType;
+import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.apache.cassandra.io.sstable.SSTableMetadata;
 import org.apache.cassandra.io.util.*;
@@ -48,6 +50,46 @@ public class CompressedRandomAccessReaderTest
 testResetAndTruncate(File.createTempFile(compressed, 1), true, 10);
 testResetAndTruncate(File.createTempFile(compressed, 2), true, 
CompressionParameters.DEFAULT_CHUNK_LENGTH);
 }
+@Test
+public void test6791() throws IOException, ConfigurationException
+{
+File f = File.createTempFile(compressed6791_, 3);
+String filename = f.getAbsolutePath();
+try
+{
+
+SSTableMetadata.Collector sstableMetadataCollector = 
SSTableMetadata.createCollector(BytesType.instance).replayPosition(null);
+CompressedSequentialWriter writer = new 
CompressedSequentialWriter(f, filename + .metadata, false, new 
CompressionParameters(SnappyCompressor.instance, 32, Collections.String, 
StringemptyMap()), sstableMetadataCollector);
+
+for (int i = 0; i  20; i++)
+writer.write(x.getBytes());
+
+FileMark mark = writer.mark();
+// write enough garbage to create new chunks:
+for (int i = 0; i  40; ++i)
+writer.write(y.getBytes());
+
+writer.resetAndTruncate(mark);
+
+for (int i = 0; i  20; i++)
+writer.write(x.getBytes());
+

[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4437bc1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4437bc1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4437bc1

Branch: refs/heads/trunk
Commit: e4437bc1881aa58b6adbf2a9483e827e37ede8d5
Parents: dcca996 41d8a5f
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 14:26:40 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 14:26:40 2014 +0100

--
 CHANGES.txt |  1 +
 .../io/compress/CompressedSequentialWriter.java |  2 +-
 .../CompressedRandomAccessReaderTest.java   | 42 
 3 files changed, 44 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4437bc1/CHANGES.txt
--
diff --cc CHANGES.txt
index 20ae300,6de11c5..516a0d1
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -36,47 -28,24 +36,48 @@@ Merged from 2.0
   * Disallow post-query re-ordering when paging (CASSANDRA-6722)
   * Fix potential paging bug with deleted columns (CASSANDRA-6748)
   * Fix NPE on BulkLoader caused by losing StreamEvent (CASSANDRA-6636)
+  * Fix truncating compression metadata (CASSANDRA-6791)
 -Merged from 1.2:
   * Add CMSClassUnloadingEnabled JVM option (CASSANDRA-6541)
   * Catch memtable flush exceptions during shutdown (CASSANDRA-6735)
 - * Fix broken streams when replacing with same IP (CASSANDRA-6622)
   * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
 - * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
 - * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)
 - * Compact hints after partial replay to clean out tombstones (CASSANDRA-)
 - * Log USING TTL/TIMESTAMP in a counter update warning (CASSANDRA-6649)
 - * Don't exchange schema between nodes with different versions 
(CASSANDRA-6695)
 - * Use real node messaging versions for schema exchange decisions 
(CASSANDRA-6700)
 - * IN on the last clustering columns + ORDER BY DESC yield no results 
(CASSANDRA-6701)
 - * Fix SecondaryIndexManager#deleteFromIndexes() (CASSANDRA-6711)
 - * Fix snapshot repair not snapshotting coordinator itself (CASSANDRA-6713)
 - * Support negative timestamps for CQL3 dates in query string (CASSANDRA-6718)
 - * Avoid NPEs when receiving table changes for an unknown keyspace 
(CASSANDRA-5631)
 - * Fix bootstrapping when there is no schema (CASSANDRA-6685)
 +
 +
 +2.1.0-beta1
 + * Add flush directory distinct from compaction directories (CASSANDRA-6357)
 + * Require JNA by default (CASSANDRA-6575)
 + * add listsnapshots command to nodetool (CASSANDRA-5742)
 + * Introduce AtomicBTreeColumns (CASSANDRA-6271, 6692)
 + * Multithreaded commitlog (CASSANDRA-3578)
 + * allocate fixed index summary memory pool and resample cold index summaries 
 +   to use less memory (CASSANDRA-5519)
 + * Removed multithreaded compaction (CASSANDRA-6142)
 + * Parallelize fetching rows for low-cardinality indexes (CASSANDRA-1337)
 + * change logging from log4j to logback (CASSANDRA-5883)
 + * switch to LZ4 compression for internode communication (CASSANDRA-5887)
 + * Stop using Thrift-generated Index* classes internally (CASSANDRA-5971)
 + * Remove 1.2 network compatibility code (CASSANDRA-5960)
 + * Remove leveled json manifest migration code (CASSANDRA-5996)
 + * Remove CFDefinition (CASSANDRA-6253)
 + * Use AtomicIntegerFieldUpdater in RefCountedMemory (CASSANDRA-6278)
 + * User-defined types for CQL3 (CASSANDRA-5590)
 + * Use of o.a.c.metrics in nodetool (CASSANDRA-5871, 6406)
 + * Batch read from OTC's queue and cleanup (CASSANDRA-1632)
 + * Secondary index support for collections (CASSANDRA-4511, 6383)
 + * SSTable metadata(Stats.db) format change (CASSANDRA-6356)
 + * Push composites support in the storage engine
 +   (CASSANDRA-5417, CASSANDRA-6520)
 + * Add snapshot space used to cfstats (CASSANDRA-6231)
 + * Add cardinality estimator for key count estimation (CASSANDRA-5906)
 + * CF id is changed to be non-deterministic. Data dir/key cache are created
 +   uniquely for CF id (CASSANDRA-5202)
 + * New counters implementation (CASSANDRA-6504)
 + * Replace UnsortedColumns, EmptyColumns, TreeMapBackedSortedColumns with new
 +   ArrayBackedSortedColumns (CASSANDRA-6630, CASSANDRA-6662, CASSANDRA-6690)
 + * Add option to use row cache with a given amount of rows (CASSANDRA-5357)
 + * Avoid repairing already repaired data (CASSANDRA-5351)
 + * Reject counter updates with USING TTL/TIMESTAMP (CASSANDRA-6649)
 + * Replace index_interval with min/max_index_interval (CASSANDRA-6379)
 + * Lift limitation that order by columns must be selected for

[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4437bc1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4437bc1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4437bc1

Branch: refs/heads/cassandra-2.1
Commit: e4437bc1881aa58b6adbf2a9483e827e37ede8d5
Parents: dcca996 41d8a5f
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 14:26:40 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 14:26:40 2014 +0100

--
 CHANGES.txt |  1 +
 .../io/compress/CompressedSequentialWriter.java |  2 +-
 .../CompressedRandomAccessReaderTest.java   | 42 
 3 files changed, 44 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4437bc1/CHANGES.txt
--
diff --cc CHANGES.txt
index 20ae300,6de11c5..516a0d1
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -36,47 -28,24 +36,48 @@@ Merged from 2.0
   * Disallow post-query re-ordering when paging (CASSANDRA-6722)
   * Fix potential paging bug with deleted columns (CASSANDRA-6748)
   * Fix NPE on BulkLoader caused by losing StreamEvent (CASSANDRA-6636)
+  * Fix truncating compression metadata (CASSANDRA-6791)
 -Merged from 1.2:
   * Add CMSClassUnloadingEnabled JVM option (CASSANDRA-6541)
   * Catch memtable flush exceptions during shutdown (CASSANDRA-6735)
 - * Fix broken streams when replacing with same IP (CASSANDRA-6622)
   * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
 - * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
 - * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)
 - * Compact hints after partial replay to clean out tombstones (CASSANDRA-)
 - * Log USING TTL/TIMESTAMP in a counter update warning (CASSANDRA-6649)
 - * Don't exchange schema between nodes with different versions 
(CASSANDRA-6695)
 - * Use real node messaging versions for schema exchange decisions 
(CASSANDRA-6700)
 - * IN on the last clustering columns + ORDER BY DESC yield no results 
(CASSANDRA-6701)
 - * Fix SecondaryIndexManager#deleteFromIndexes() (CASSANDRA-6711)
 - * Fix snapshot repair not snapshotting coordinator itself (CASSANDRA-6713)
 - * Support negative timestamps for CQL3 dates in query string (CASSANDRA-6718)
 - * Avoid NPEs when receiving table changes for an unknown keyspace 
(CASSANDRA-5631)
 - * Fix bootstrapping when there is no schema (CASSANDRA-6685)
 +
 +
 +2.1.0-beta1
 + * Add flush directory distinct from compaction directories (CASSANDRA-6357)
 + * Require JNA by default (CASSANDRA-6575)
 + * add listsnapshots command to nodetool (CASSANDRA-5742)
 + * Introduce AtomicBTreeColumns (CASSANDRA-6271, 6692)
 + * Multithreaded commitlog (CASSANDRA-3578)
 + * allocate fixed index summary memory pool and resample cold index summaries 
 +   to use less memory (CASSANDRA-5519)
 + * Removed multithreaded compaction (CASSANDRA-6142)
 + * Parallelize fetching rows for low-cardinality indexes (CASSANDRA-1337)
 + * change logging from log4j to logback (CASSANDRA-5883)
 + * switch to LZ4 compression for internode communication (CASSANDRA-5887)
 + * Stop using Thrift-generated Index* classes internally (CASSANDRA-5971)
 + * Remove 1.2 network compatibility code (CASSANDRA-5960)
 + * Remove leveled json manifest migration code (CASSANDRA-5996)
 + * Remove CFDefinition (CASSANDRA-6253)
 + * Use AtomicIntegerFieldUpdater in RefCountedMemory (CASSANDRA-6278)
 + * User-defined types for CQL3 (CASSANDRA-5590)
 + * Use of o.a.c.metrics in nodetool (CASSANDRA-5871, 6406)
 + * Batch read from OTC's queue and cleanup (CASSANDRA-1632)
 + * Secondary index support for collections (CASSANDRA-4511, 6383)
 + * SSTable metadata(Stats.db) format change (CASSANDRA-6356)
 + * Push composites support in the storage engine
 +   (CASSANDRA-5417, CASSANDRA-6520)
 + * Add snapshot space used to cfstats (CASSANDRA-6231)
 + * Add cardinality estimator for key count estimation (CASSANDRA-5906)
 + * CF id is changed to be non-deterministic. Data dir/key cache are created
 +   uniquely for CF id (CASSANDRA-5202)
 + * New counters implementation (CASSANDRA-6504)
 + * Replace UnsortedColumns, EmptyColumns, TreeMapBackedSortedColumns with new
 +   ArrayBackedSortedColumns (CASSANDRA-6630, CASSANDRA-6662, CASSANDRA-6690)
 + * Add option to use row cache with a given amount of rows (CASSANDRA-5357)
 + * Avoid repairing already repaired data (CASSANDRA-5351)
 + * Reject counter updates with USING TTL/TIMESTAMP (CASSANDRA-6649)
 + * Replace index_interval with min/max_index_interval (CASSANDRA-6379)
 + * Lift limitation that order by columns must be

[1/2] git commit: Fix resetAndTruncate:ing CompressionMetadata

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 dcca99684 - e4437bc18


Fix resetAndTruncate:ing CompressionMetadata

Patch by kvaster, reviewed by marcuse for CASSANDRA-6791


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/41d8a5f4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/41d8a5f4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/41d8a5f4

Branch: refs/heads/cassandra-2.1
Commit: 41d8a5f4861c37ff8e22344418e9277236672c1f
Parents: b3a9a44
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 11:35:03 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 14:21:00 2014 +0100

--
 CHANGES.txt |  1 +
 .../io/compress/CompressedSequentialWriter.java |  2 +-
 .../CompressedRandomAccessReaderTest.java   | 42 
 3 files changed, 44 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/41d8a5f4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3e73f91..6de11c5 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -28,6 +28,7 @@
  * Disallow post-query re-ordering when paging (CASSANDRA-6722)
  * Fix potential paging bug with deleted columns (CASSANDRA-6748)
  * Fix NPE on BulkLoader caused by losing StreamEvent (CASSANDRA-6636)
+ * Fix truncating compression metadata (CASSANDRA-6791)
 Merged from 1.2:
  * Add CMSClassUnloadingEnabled JVM option (CASSANDRA-6541)
  * Catch memtable flush exceptions during shutdown (CASSANDRA-6735)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/41d8a5f4/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java 
b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
index 54b990f..eef5b17 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
@@ -231,7 +231,7 @@ public class CompressedSequentialWriter extends 
SequentialWriter
 
 // truncate data and index file
 truncate(chunkOffset);
-metadataWriter.resetAndTruncate(realMark.nextChunkIndex);
+metadataWriter.resetAndTruncate(realMark.nextChunkIndex - 1);
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/41d8a5f4/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index ee32a0e..3c9dfe5 100644
--- 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++ 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@ -19,11 +19,13 @@
 package org.apache.cassandra.io.compress;
 
 import java.io.*;
+import java.util.Collections;
 import java.util.Random;
 
 import org.junit.Test;
 
 import org.apache.cassandra.db.marshal.BytesType;
+import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.apache.cassandra.io.sstable.SSTableMetadata;
 import org.apache.cassandra.io.util.*;
@@ -48,6 +50,46 @@ public class CompressedRandomAccessReaderTest
 testResetAndTruncate(File.createTempFile(compressed, 1), true, 10);
 testResetAndTruncate(File.createTempFile(compressed, 2), true, 
CompressionParameters.DEFAULT_CHUNK_LENGTH);
 }
+@Test
+public void test6791() throws IOException, ConfigurationException
+{
+File f = File.createTempFile(compressed6791_, 3);
+String filename = f.getAbsolutePath();
+try
+{
+
+SSTableMetadata.Collector sstableMetadataCollector = 
SSTableMetadata.createCollector(BytesType.instance).replayPosition(null);
+CompressedSequentialWriter writer = new 
CompressedSequentialWriter(f, filename + .metadata, false, new 
CompressionParameters(SnappyCompressor.instance, 32, Collections.String, 
StringemptyMap()), sstableMetadataCollector);
+
+for (int i = 0; i  20; i++)
+writer.write(x.getBytes());
+
+FileMark mark = writer.mark();
+// write enough garbage to create new chunks:
+for (int i = 0; i  40; ++i)
+writer.write(y.getBytes());
+
+writer.resetAndTruncate(mark);
+
+for (int i = 0; i  20; i++)
+

[jira] [Commented] (CASSANDRA-6285) LCS compaction failing with Exception

2014-03-03 Thread Nikolai Grigoriev (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918048#comment-13918048
 ] 

Nikolai Grigoriev commented on CASSANDRA-6285:
--

[~krummas]

I think using HSHA makes it easier to reproduce but...I am running SYNC for 
over a week now and recently I have experienced the same issue again.

We had another unclean shutdown (hrrr...some people are smarter than the UPSes 
;) ) and after bringing the nodes back I have found that  on one node my 
compactions constantly fail with FileNotFoundException. Even worse, I can't 
scrub the keyspace/CF in question because scrub fails instantly with 
RuntimeException: Tried to hard link to file that does not exist I have 
reported that one too. It is impossible to scrub. The only way to fix that 
issue I have found so far is to restart Cassandra on that node, stop 
compactions as soon as it starts (well, I could disable them differently, I 
assume) and then scrub. Sometimes I have to do it in several iterations to 
complete the process. Once I scrub all problematic KS/CFs I see no more 
exceptions.

 LCS compaction failing with Exception
 -

 Key: CASSANDRA-6285
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6285
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2
Reporter: David Sauer
Assignee: Marcus Eriksson
 Fix For: 2.0.6

 Attachments: compaction_test.py


 After altering everything to LCS the table OpsCenter.rollups60 amd one other 
 none OpsCenter-Table got stuck with everything hanging around in L0.
 The compaction started and ran until the logs showed this:
 ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java 
 (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime]
 java.lang.RuntimeException: Last written key 
 DecoratedKey(1326283851463420237, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564)
  = current key DecoratedKey(954210699457429663, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f)
  writing into 
 /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
   at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:724)
 Moving back to STC worked to keep the compactions running.
 Especialy my own Table i would like to move to LCS.
 After a major compaction with STC the move to LCS fails with the same 
 Exception.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Fix merge

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 e4437bc18 - 043953127


Fix merge


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/04395312
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/04395312
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/04395312

Branch: refs/heads/cassandra-2.1
Commit: 04395312745805ed7e4fd201b88bcd3d46b3a601
Parents: e4437bc
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 14:39:40 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 14:39:40 2014 +0100

--
 .../cassandra/io/compress/CompressedRandomAccessReaderTest.java   | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/04395312/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index c9ddfea..f8fcf76 100644
--- 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++ 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@ -29,6 +29,7 @@ import org.apache.cassandra.db.marshal.BytesType;
 import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.apache.cassandra.io.sstable.metadata.MetadataCollector;
+import org.apache.cassandra.io.sstable.metadata.StatsMetadata;
 import org.apache.cassandra.io.util.*;
 
 import static org.junit.Assert.assertEquals;
@@ -59,7 +60,7 @@ public class CompressedRandomAccessReaderTest
 try
 {
 
-SSTableMetadata.Collector sstableMetadataCollector = 
SSTableMetadata.createCollector(BytesType.instance).replayPosition(null);
+MetadataCollector sstableMetadataCollector = new 
MetadataCollector(new SimpleDenseCellNameType(BytesType.instance));
 CompressedSequentialWriter writer = new 
CompressedSequentialWriter(f, filename + .metadata, false, new 
CompressionParameters(SnappyCompressor.instance, 32, Collections.String, 
StringemptyMap()), sstableMetadataCollector);
 
 for (int i = 0; i  20; i++)

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2c8100af
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2c8100af
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2c8100af

Branch: refs/heads/trunk
Commit: 2c8100af210990bf1f15cc6691066787b88e5f25
Parents: e80564f 0439531
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 14:39:45 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 14:39:45 2014 +0100

--
 .../cassandra/io/compress/CompressedRandomAccessReaderTest.java   | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--

[1/2] git commit: Fix merge

Repository: cassandra
Updated Branches:
  refs/heads/trunk e80564f39 - 2c8100af2


Fix merge


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/04395312
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/04395312
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/04395312

Branch: refs/heads/trunk
Commit: 04395312745805ed7e4fd201b88bcd3d46b3a601
Parents: e4437bc
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 14:39:40 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 14:39:40 2014 +0100

--
 .../cassandra/io/compress/CompressedRandomAccessReaderTest.java   | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/04395312/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index c9ddfea..f8fcf76 100644
--- 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++ 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@ -29,6 +29,7 @@ import org.apache.cassandra.db.marshal.BytesType;
 import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.apache.cassandra.io.sstable.metadata.MetadataCollector;
+import org.apache.cassandra.io.sstable.metadata.StatsMetadata;
 import org.apache.cassandra.io.util.*;
 
 import static org.junit.Assert.assertEquals;
@@ -59,7 +60,7 @@ public class CompressedRandomAccessReaderTest
 try
 {
 
-SSTableMetadata.Collector sstableMetadataCollector = 
SSTableMetadata.createCollector(BytesType.instance).replayPosition(null);
+MetadataCollector sstableMetadataCollector = new 
MetadataCollector(new SimpleDenseCellNameType(BytesType.instance));
 CompressedSequentialWriter writer = new 
CompressedSequentialWriter(f, filename + .metadata, false, new 
CompressionParameters(SnappyCompressor.instance, 32, Collections.String, 
StringemptyMap()), sstableMetadataCollector);
 
 for (int i = 0; i  20; i++)

[jira] [Commented] (CASSANDRA-6285) LCS compaction failing with Exception


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918053#comment-13918053
 ] 

Viktor Kuzmin commented on CASSANDRA-6285:
--

Bug with FileNotFoundException is not related to HsHa problem.
And about several iterations for scrub:
https://issues.apache.org/jira/browse/CASSANDRA-6791


 LCS compaction failing with Exception
 -

 Key: CASSANDRA-6285
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6285
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2
Reporter: David Sauer
Assignee: Marcus Eriksson
 Fix For: 2.0.6

 Attachments: compaction_test.py


 After altering everything to LCS the table OpsCenter.rollups60 amd one other 
 none OpsCenter-Table got stuck with everything hanging around in L0.
 The compaction started and ran until the logs showed this:
 ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java 
 (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime]
 java.lang.RuntimeException: Last written key 
 DecoratedKey(1326283851463420237, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564)
  = current key DecoratedKey(954210699457429663, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f)
  writing into 
 /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
   at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:724)
 Moving back to STC worked to keep the compactions running.
 Especialy my own Table i would like to move to LCS.
 After a major compaction with STC the move to LCS fails with the same 
 Exception.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6745) Require specifying rows_per_partition_to_cache


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918057#comment-13918057
 ] 

Sylvain Lebresne commented on CASSANDRA-6745:
-

Hum, I kind of like my version more :). Basically, my goal is to make things as 
future proof as possible since it's not at all unlikely that in the future we 
might add per-table setting for both cache. So my idea was to have a simple 
on/off option switch for each cache (and I kind of like for that to be a 
boolean) + any number of options that control the behavior. Typically, having 
rows_per_partition being the option that control whether the row cache is 
enabled or not feels weird to me, feeling that we're trying to cram too much 
info in that option. But I don't know, maybe it's just me trying to justify 
what is ultimately a personal preference?

Other than that, I do would rather not change the thrift interface. We 
definitively can't change an existing option like caching like that as this 
would break clients which we don't do for thrift, and while we could probably 
deprecate 'caching' to replace it with some 'caching_options', this would still 
be borderline in term of backward compatibility, so I propose to leave things 
as they are. In fact, I would even argue that this whole issue is kind of 
breaking if done for thrift and so we may want to stick to CQL only here.

 Require specifying rows_per_partition_to_cache
 --

 Key: CASSANDRA-6745
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6745
 Project: Cassandra
  Issue Type: Bug
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
Priority: Trivial
 Fix For: 2.1 beta2


 We should require specifying rows_to_cache_per_partition for new tables or 
 newly ALTERed when row caching is enabled.
 Pre-upgrade should be grandfathered in as ALL to match existing semantics.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Fix resetAndTruncate:ing CompressionMetadata

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-1.2 f08ae394f - 64098f7d6


Fix resetAndTruncate:ing CompressionMetadata

Patch by kvaster, reviewed by marcuse for CASSANDRA-6791


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/64098f7d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/64098f7d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/64098f7d

Branch: refs/heads/cassandra-1.2
Commit: 64098f7d6f0b122448693a3c6da16af54c99013b
Parents: f08ae39
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 15:03:34 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 15:03:34 2014 +0100

--
 CHANGES.txt |  2 +-
 .../io/compress/CompressedSequentialWriter.java |  2 +-
 .../CompressedRandomAccessReaderTest.java   | 43 
 3 files changed, 45 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/64098f7d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 780b528..b3c0a35 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -19,7 +19,7 @@
  * Support negative timestamps for CQL3 dates in query string (CASSANDRA-6718)
  * Avoid NPEs when receiving table changes for an unknown keyspace 
(CASSANDRA-5631)
  * Fix bootstrapping when there is no schema (CASSANDRA-6685)
-
+ * Fix truncating compression metadata (CASSANDRA-6791)
 
 1.2.15
  * Move handling of migration event source to solve bootstrap race 
(CASSANDRA-6648)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/64098f7d/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java 
b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
index 00eb5a7..da55e83 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
@@ -231,7 +231,7 @@ public class CompressedSequentialWriter extends 
SequentialWriter
 
 // truncate data and index file
 truncate(chunkOffset);
-metadataWriter.resetAndTruncate(realMark.nextChunkIndex);
+metadataWriter.resetAndTruncate(realMark.nextChunkIndex - 1);
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/64098f7d/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index 830c3e1..678a650 100644
--- 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++ 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@ -19,10 +19,12 @@
 package org.apache.cassandra.io.compress;
 
 import java.io.*;
+import java.util.Collections;
 import java.util.Random;
 
 import org.junit.Test;
 
+import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.apache.cassandra.io.sstable.SSTableMetadata;
 import org.apache.cassandra.io.util.*;
@@ -95,6 +97,47 @@ public class CompressedRandomAccessReaderTest
 }
 
 @Test
+public void test6791() throws IOException, ConfigurationException
+{
+File f = File.createTempFile(compressed6791_, 3);
+String filename = f.getAbsolutePath();
+try
+{
+
+SSTableMetadata.Collector sstableMetadataCollector = 
SSTableMetadata.createCollector().replayPosition(null);
+CompressedSequentialWriter writer = new 
CompressedSequentialWriter(f, filename + .metadata, false, new 
CompressionParameters(SnappyCompressor.instance, 32, Collections.String, 
StringemptyMap()), sstableMetadataCollector);
+
+for (int i = 0; i  20; i++)
+writer.write(x.getBytes());
+
+FileMark mark = writer.mark();
+// write enough garbage to create new chunks:
+for (int i = 0; i  40; i++)
+writer.write(y.getBytes());
+
+writer.resetAndTruncate(mark);
+
+for (int i = 0; i  20; i++)
+writer.write(x.getBytes());
+writer.close();
+
+CompressedRandomAccessReader reader = 
CompressedRandomAccessReader.open(filename, new CompressionMetadata(filename + 
.metadata, f.length()), false);
+String res = reader.readLine();
+assertEquals(res,

[2/2] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

Merge branch 'cassandra-1.2' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ef20671c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ef20671c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ef20671c

Branch: refs/heads/cassandra-2.0
Commit: ef20671c2c99ab40362785f4718c9e2c13edbda2
Parents: 41d8a5f 64098f7
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 15:04:00 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 15:04:00 2014 +0100

--

--

[4/4] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db6b563f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db6b563f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db6b563f

Branch: refs/heads/trunk
Commit: db6b563f143dc5f2037244854527cafc8594e730
Parents: 2c8100a 56631e1
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 15:04:17 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 15:04:17 2014 +0100

--

--

[2/4] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

Merge branch 'cassandra-1.2' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ef20671c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ef20671c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ef20671c

Branch: refs/heads/trunk
Commit: ef20671c2c99ab40362785f4718c9e2c13edbda2
Parents: 41d8a5f 64098f7
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 15:04:00 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 15:04:00 2014 +0100

--

--

[3/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/56631e13
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/56631e13
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/56631e13

Branch: refs/heads/cassandra-2.1
Commit: 56631e139092d81d55ea8c7c7ddccb3ff16eef62
Parents: 0439531 ef20671
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 15:04:10 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 15:04:10 2014 +0100

--

--

[3/4] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/56631e13
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/56631e13
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/56631e13

Branch: refs/heads/trunk
Commit: 56631e139092d81d55ea8c7c7ddccb3ff16eef62
Parents: 0439531 ef20671
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 15:04:10 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 15:04:10 2014 +0100

--

--

[1/2] git commit: Fix resetAndTruncate:ing CompressionMetadata

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 41d8a5f48 - ef20671c2


Fix resetAndTruncate:ing CompressionMetadata

Patch by kvaster, reviewed by marcuse for CASSANDRA-6791


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/64098f7d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/64098f7d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/64098f7d

Branch: refs/heads/cassandra-2.0
Commit: 64098f7d6f0b122448693a3c6da16af54c99013b
Parents: f08ae39
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 15:03:34 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 15:03:34 2014 +0100

--
 CHANGES.txt |  2 +-
 .../io/compress/CompressedSequentialWriter.java |  2 +-
 .../CompressedRandomAccessReaderTest.java   | 43 
 3 files changed, 45 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/64098f7d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 780b528..b3c0a35 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -19,7 +19,7 @@
  * Support negative timestamps for CQL3 dates in query string (CASSANDRA-6718)
  * Avoid NPEs when receiving table changes for an unknown keyspace 
(CASSANDRA-5631)
  * Fix bootstrapping when there is no schema (CASSANDRA-6685)
-
+ * Fix truncating compression metadata (CASSANDRA-6791)
 
 1.2.15
  * Move handling of migration event source to solve bootstrap race 
(CASSANDRA-6648)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/64098f7d/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java 
b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
index 00eb5a7..da55e83 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
@@ -231,7 +231,7 @@ public class CompressedSequentialWriter extends 
SequentialWriter
 
 // truncate data and index file
 truncate(chunkOffset);
-metadataWriter.resetAndTruncate(realMark.nextChunkIndex);
+metadataWriter.resetAndTruncate(realMark.nextChunkIndex - 1);
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/64098f7d/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index 830c3e1..678a650 100644
--- 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++ 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@ -19,10 +19,12 @@
 package org.apache.cassandra.io.compress;
 
 import java.io.*;
+import java.util.Collections;
 import java.util.Random;
 
 import org.junit.Test;
 
+import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.apache.cassandra.io.sstable.SSTableMetadata;
 import org.apache.cassandra.io.util.*;
@@ -95,6 +97,47 @@ public class CompressedRandomAccessReaderTest
 }
 
 @Test
+public void test6791() throws IOException, ConfigurationException
+{
+File f = File.createTempFile(compressed6791_, 3);
+String filename = f.getAbsolutePath();
+try
+{
+
+SSTableMetadata.Collector sstableMetadataCollector = 
SSTableMetadata.createCollector().replayPosition(null);
+CompressedSequentialWriter writer = new 
CompressedSequentialWriter(f, filename + .metadata, false, new 
CompressionParameters(SnappyCompressor.instance, 32, Collections.String, 
StringemptyMap()), sstableMetadataCollector);
+
+for (int i = 0; i  20; i++)
+writer.write(x.getBytes());
+
+FileMark mark = writer.mark();
+// write enough garbage to create new chunks:
+for (int i = 0; i  40; i++)
+writer.write(y.getBytes());
+
+writer.resetAndTruncate(mark);
+
+for (int i = 0; i  20; i++)
+writer.write(x.getBytes());
+writer.close();
+
+CompressedRandomAccessReader reader = 
CompressedRandomAccessReader.open(filename, new CompressionMetadata(filename + 
.metadata, f.length()), false);
+String res = reader.readLine();
+assertEquals(res,

[1/3] git commit: Fix resetAndTruncate:ing CompressionMetadata

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 043953127 - 56631e139


Fix resetAndTruncate:ing CompressionMetadata

Patch by kvaster, reviewed by marcuse for CASSANDRA-6791


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/64098f7d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/64098f7d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/64098f7d

Branch: refs/heads/cassandra-2.1
Commit: 64098f7d6f0b122448693a3c6da16af54c99013b
Parents: f08ae39
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 15:03:34 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 15:03:34 2014 +0100

--
 CHANGES.txt |  2 +-
 .../io/compress/CompressedSequentialWriter.java |  2 +-
 .../CompressedRandomAccessReaderTest.java   | 43 
 3 files changed, 45 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/64098f7d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 780b528..b3c0a35 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -19,7 +19,7 @@
  * Support negative timestamps for CQL3 dates in query string (CASSANDRA-6718)
  * Avoid NPEs when receiving table changes for an unknown keyspace 
(CASSANDRA-5631)
  * Fix bootstrapping when there is no schema (CASSANDRA-6685)
-
+ * Fix truncating compression metadata (CASSANDRA-6791)
 
 1.2.15
  * Move handling of migration event source to solve bootstrap race 
(CASSANDRA-6648)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/64098f7d/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java 
b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
index 00eb5a7..da55e83 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
@@ -231,7 +231,7 @@ public class CompressedSequentialWriter extends 
SequentialWriter
 
 // truncate data and index file
 truncate(chunkOffset);
-metadataWriter.resetAndTruncate(realMark.nextChunkIndex);
+metadataWriter.resetAndTruncate(realMark.nextChunkIndex - 1);
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/64098f7d/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index 830c3e1..678a650 100644
--- 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++ 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@ -19,10 +19,12 @@
 package org.apache.cassandra.io.compress;
 
 import java.io.*;
+import java.util.Collections;
 import java.util.Random;
 
 import org.junit.Test;
 
+import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.apache.cassandra.io.sstable.SSTableMetadata;
 import org.apache.cassandra.io.util.*;
@@ -95,6 +97,47 @@ public class CompressedRandomAccessReaderTest
 }
 
 @Test
+public void test6791() throws IOException, ConfigurationException
+{
+File f = File.createTempFile(compressed6791_, 3);
+String filename = f.getAbsolutePath();
+try
+{
+
+SSTableMetadata.Collector sstableMetadataCollector = 
SSTableMetadata.createCollector().replayPosition(null);
+CompressedSequentialWriter writer = new 
CompressedSequentialWriter(f, filename + .metadata, false, new 
CompressionParameters(SnappyCompressor.instance, 32, Collections.String, 
StringemptyMap()), sstableMetadataCollector);
+
+for (int i = 0; i  20; i++)
+writer.write(x.getBytes());
+
+FileMark mark = writer.mark();
+// write enough garbage to create new chunks:
+for (int i = 0; i  40; i++)
+writer.write(y.getBytes());
+
+writer.resetAndTruncate(mark);
+
+for (int i = 0; i  20; i++)
+writer.write(x.getBytes());
+writer.close();
+
+CompressedRandomAccessReader reader = 
CompressedRandomAccessReader.open(filename, new CompressionMetadata(filename + 
.metadata, f.length()), false);
+String res = reader.readLine();
+assertEquals(res,

[1/4] git commit: Fix resetAndTruncate:ing CompressionMetadata

Repository: cassandra
Updated Branches:
  refs/heads/trunk 2c8100af2 - db6b563f1


Fix resetAndTruncate:ing CompressionMetadata

Patch by kvaster, reviewed by marcuse for CASSANDRA-6791


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/64098f7d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/64098f7d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/64098f7d

Branch: refs/heads/trunk
Commit: 64098f7d6f0b122448693a3c6da16af54c99013b
Parents: f08ae39
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 15:03:34 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 15:03:34 2014 +0100

--
 CHANGES.txt |  2 +-
 .../io/compress/CompressedSequentialWriter.java |  2 +-
 .../CompressedRandomAccessReaderTest.java   | 43 
 3 files changed, 45 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/64098f7d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 780b528..b3c0a35 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -19,7 +19,7 @@
  * Support negative timestamps for CQL3 dates in query string (CASSANDRA-6718)
  * Avoid NPEs when receiving table changes for an unknown keyspace 
(CASSANDRA-5631)
  * Fix bootstrapping when there is no schema (CASSANDRA-6685)
-
+ * Fix truncating compression metadata (CASSANDRA-6791)
 
 1.2.15
  * Move handling of migration event source to solve bootstrap race 
(CASSANDRA-6648)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/64098f7d/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java 
b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
index 00eb5a7..da55e83 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
@@ -231,7 +231,7 @@ public class CompressedSequentialWriter extends 
SequentialWriter
 
 // truncate data and index file
 truncate(chunkOffset);
-metadataWriter.resetAndTruncate(realMark.nextChunkIndex);
+metadataWriter.resetAndTruncate(realMark.nextChunkIndex - 1);
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/64098f7d/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
index 830c3e1..678a650 100644
--- 
a/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
+++ 
b/test/unit/org/apache/cassandra/io/compress/CompressedRandomAccessReaderTest.java
@@ -19,10 +19,12 @@
 package org.apache.cassandra.io.compress;
 
 import java.io.*;
+import java.util.Collections;
 import java.util.Random;
 
 import org.junit.Test;
 
+import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.apache.cassandra.io.sstable.SSTableMetadata;
 import org.apache.cassandra.io.util.*;
@@ -95,6 +97,47 @@ public class CompressedRandomAccessReaderTest
 }
 
 @Test
+public void test6791() throws IOException, ConfigurationException
+{
+File f = File.createTempFile(compressed6791_, 3);
+String filename = f.getAbsolutePath();
+try
+{
+
+SSTableMetadata.Collector sstableMetadataCollector = 
SSTableMetadata.createCollector().replayPosition(null);
+CompressedSequentialWriter writer = new 
CompressedSequentialWriter(f, filename + .metadata, false, new 
CompressionParameters(SnappyCompressor.instance, 32, Collections.String, 
StringemptyMap()), sstableMetadataCollector);
+
+for (int i = 0; i  20; i++)
+writer.write(x.getBytes());
+
+FileMark mark = writer.mark();
+// write enough garbage to create new chunks:
+for (int i = 0; i  40; i++)
+writer.write(y.getBytes());
+
+writer.resetAndTruncate(mark);
+
+for (int i = 0; i  20; i++)
+writer.write(x.getBytes());
+writer.close();
+
+CompressedRandomAccessReader reader = 
CompressedRandomAccessReader.open(filename, new CompressionMetadata(filename + 
.metadata, f.length()), false);
+String res = reader.readLine();
+assertEquals(res,

[2/3] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

Merge branch 'cassandra-1.2' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ef20671c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ef20671c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ef20671c

Branch: refs/heads/cassandra-2.1
Commit: ef20671c2c99ab40362785f4718c9e2c13edbda2
Parents: 41d8a5f 64098f7
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Mar 3 15:04:00 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Mar 3 15:04:00 2014 +0100

--

--

[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-03-03 Thread Anne Sullivan (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918081#comment-13918081
 ] 

Anne Sullivan commented on CASSANDRA-:
--

With this fix, we're seeing a case where the user-defined compaction that runs 
post-hint-replay is not removing the sstables as expected.  The hinted handoff 
finishes successfully, but some sstables remain (100% tombstones) and then the 
10 minute check for hints keeps trying to start HH and failing with the 
tombstone exception.  Running a manual compaction clears them up.  It occurs 
100% of the time if we have some larger sstables in the hints CF ... so 4 small 
sstables ~ 10MB and 2 sstables  100MB, the 2 that are large are never 
processed by the user-defined compaction.  


 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-5989) java.lang.OutOfMemoryError: Requested array size exceeds VM limit


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918131#comment-13918131
 ] 

Jonathan Ellis commented on CASSANDRA-5989:
---

The general I'm requesting really large resultsets problem is fixed by 
CASSANDRA-4415 in any case.

(Corollary: there is no way to fix this server side for Thrift, it's the 
client's job to exercise discretion.)

 java.lang.OutOfMemoryError: Requested array size exceeds VM limit
 -

 Key: CASSANDRA-5989
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5989
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra 1.2.8
 Oracle Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
 RHEL6
Reporter: Karl Mueller

 This occurred in one of our nodes today. I don't have any helpful information 
 on what is going on beforehand yet - logs don't have anything I could see 
 that's tied for sure to it.
 A few things happened in the logs beforehand. A little bit of standard GC, a 
 bunch of status-logger entries 10 minutes before the crash, and a few nodes 
 going up and down on the gossip.
 ERROR [Thrift:7495] 2013-09-03 11:01:12,486 CassandraDaemon.java (line 192) 
 Exception in thread Thread[Thrift:7495,5,main]
 java.lang.OutOfMemoryError: Requested array size exceeds VM limit
 at java.util.Arrays.copyOf(Arrays.java:2271)
 at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
 at 
 java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
 at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
 at 
 org.apache.thrift.transport.TFramedTransport.write(TFramedTransport.java:146)
 at 
 org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:163)
 at 
 org.apache.cassandra.thrift.TBinaryProtocol.writeBinary(TBinaryProtocol.java:69)
 at org.apache.cassandra.thrift.Column.write(Column.java:579)
 at org.apache.cassandra.thrift.CqlRow.write(CqlRow.java:439)
 at org.apache.cassandra.thrift.CqlResult.write(CqlResult.java:602)
 at 
 org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.write(Cassandra.java:37895)
 at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:34)
 at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
 at 
 org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:199)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:724)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6632) Rename row cache to partition cache


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918135#comment-13918135
 ] 

Jonathan Ellis commented on CASSANDRA-6632:
---

+1

 Rename row cache to partition cache
 ---

 Key: CASSANDRA-6632
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6632
 Project: Cassandra
  Issue Type: Improvement
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson
Priority: Minor
 Fix For: 2.1 beta2


 We should rename row cache into partition cache to avoid confusion.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6794) Optimise slab allocator to enable higher number of column families


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-6794:
--

Assignee: (was: Benedict)

 Optimise slab allocator to enable higher number of column families
 --

 Key: CASSANDRA-6794
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6794
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jeremy Hanna
Priority: Minor

 Currently the slab allocator allocates 1MB per column family.  This has been 
 very beneficial for gc efficiency.  However, it makes it more difficult to 
 have large numbers of column families.
 It would be preferable to have a more intelligent way to allocate slabs so 
 that there is more flexibility between slab allocator and non-slab allocator 
 behaviour.
 A simple first step is to ramp up size of slabs from small (say  8KB) when 
 empty, to 1MB after a few slabs.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-6285:
--

Since Version: 2.0.0
 Priority: Critical  (was: Major)
 Assignee: Pavel Yaskevich  (was: Marcus Eriksson)
  Summary: 2.0 HSHA server introduces corrupt data  (was: LCS 
compaction failing with Exception)

According to 
http://mail-archives.apache.org/mod_mbox/cassandra-user/201402.mbox/%3C038601cf28ea$a2e504d0$e8af0e70$@struq.com%3E
 the 0.9 TThreadSelectorServer works well, although I'm not sure if he means 
that it performs better than 0.8 TTSS or just that it doesn't cause corruption. 
:)

 2.0 HSHA server introduces corrupt data
 ---

 Key: CASSANDRA-6285
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6285
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2
Reporter: David Sauer
Assignee: Pavel Yaskevich
Priority: Critical
 Fix For: 2.0.6

 Attachments: compaction_test.py


 After altering everything to LCS the table OpsCenter.rollups60 amd one other 
 none OpsCenter-Table got stuck with everything hanging around in L0.
 The compaction started and ran until the logs showed this:
 ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java 
 (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime]
 java.lang.RuntimeException: Last written key 
 DecoratedKey(1326283851463420237, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564)
  = current key DecoratedKey(954210699457429663, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f)
  writing into 
 /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
   at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:724)
 Moving back to STC worked to keep the compactions running.
 Especialy my own Table i would like to move to LCS.
 After a major compaction with STC the move to LCS fails with the same 
 Exception.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-6736) Windows7 AccessDeniedException on commit log


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918170#comment-13918170
 ] 

Joshua McKenzie edited comment on CASSANDRA-6736 at 3/3/14 3:44 PM:


I've reproduced that error with scrubDataDirectories locally as well - I'll 
create a new JIRA out of that since it's separate from the commit log issue.  
Historically when errors like this popped up, repeatedly attempting to start C* 
eventually resolved it and I've seen that locally on my system as well.

rejectedException from a ThreadPoolExecutor after shutdown is expected 
behavior.  There's been some discussion on other tickets about handling that 
more gracefully but the general consensus seems to be that it's Working As 
Intended, though noisy.

The SSTableDeletingTask issue you saw after exhausting your heap and restarting 
may be related to CASSANDRA-6283 as we're seeing the same stack there, though 
from different circumstances.


was (Author: joshuamckenzie):
I've reproduced that error with scrubDataDirectories locally as well - I'll 
create a new JIRA out of that since it's separate from the commit log issue.  
Historically when errors like this popped up, repeatedly attempting to start C* 
eventually resolved it and I've seen that locally on my system as well.

rejectedException from a ThreadPoolExecutor after shutdown is expected 
behavior.  There's been some discussion on other tickets about handling that 
more gracefully but the general consensus seems to be that it's Working As 
Intended, though noisy.

The SSTableDeletingTask issue you saw after exhausting your heap and restarting 
may be related to CASSANDRA-6283 as we're seeing the same there.

 Windows7 AccessDeniedException on commit log 
 -

 Key: CASSANDRA-6736
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6736
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Windows 7, quad core, 8GB RAM, single Cassandra node, 
 Cassandra 2.0.5 with leakdetect patch from CASSANDRA-6283
Reporter: Bill Mitchell
Assignee: Joshua McKenzie
 Attachments: 2014-02-18-22-16.log


 Similar to the data file deletion of CASSANDRA-6283, under heavy load with 
 logged batches, I am seeing a problem where the Commit log cannot be deleted:
  ERROR [COMMIT-LOG-ALLOCATOR] 2014-02-18 22:15:58,252 CassandraDaemon.java 
 (line 192) Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
  FSWriteError in C:\Program Files\DataStax 
 Community\data\commitlog\CommitLog-3-1392761510706.log
   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:120)
   at 
 org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:150)
   at 
 org.apache.cassandra.db.commitlog.CommitLogAllocator$4.run(CommitLogAllocator.java:217)
   at 
 org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at java.lang.Thread.run(Unknown Source)
 Caused by: java.nio.file.AccessDeniedException: C:\Program Files\DataStax 
 Community\data\commitlog\CommitLog-3-1392761510706.log
   at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
   at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source)
   at sun.nio.fs.AbstractFileSystemProvider.delete(Unknown Source)
   at java.nio.file.Files.delete(Unknown Source)
   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:116)
   ... 5 more
 (Attached in 2014-02-18-22-16.log is a larger excerpt from the cassandra.log.)
 In this particular case, I was trying to do 100 million inserts into two 
 tables in parallel, one with a single wide row and one with narrow rows, and 
 the error appeared after inserting 43,151,232 rows.  So it does take a while 
 to trip over this timing issue.  
 It may be aggravated by the size of the batches. This test was writing 10,000 
 rows to each table in a batch.  
 When I try switching the same test from using a logged batch to an unlogged 
 batch, and no such failure appears. So the issue could be related to the use 
 of large, logged batches, or it could be that unlogged batches just change 
 the probability of failure.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6736) Windows7 AccessDeniedException on commit log


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918170#comment-13918170
 ] 

Joshua McKenzie commented on CASSANDRA-6736:


I've reproduced that error with scrubDataDirectories locally as well - I'll 
create a new JIRA out of that since it's separate from the commit log issue.  
Historically when errors like this popped up, repeatedly attempting to start C* 
eventually resolved it and I've seen that locally on my system as well.

rejectedException from a ThreadPoolExecutor after shutdown is expected 
behavior.  There's been some discussion on other tickets about handling that 
more gracefully but the general consensus seems to be that it's Working As 
Intended, though noisy.

The SSTableDeletingTask issue you saw after exhausting your heap and restarting 
may be related to CASSANDRA-6283 as we're seeing the same there.

 Windows7 AccessDeniedException on commit log 
 -

 Key: CASSANDRA-6736
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6736
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Windows 7, quad core, 8GB RAM, single Cassandra node, 
 Cassandra 2.0.5 with leakdetect patch from CASSANDRA-6283
Reporter: Bill Mitchell
Assignee: Joshua McKenzie
 Attachments: 2014-02-18-22-16.log


 Similar to the data file deletion of CASSANDRA-6283, under heavy load with 
 logged batches, I am seeing a problem where the Commit log cannot be deleted:
  ERROR [COMMIT-LOG-ALLOCATOR] 2014-02-18 22:15:58,252 CassandraDaemon.java 
 (line 192) Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
  FSWriteError in C:\Program Files\DataStax 
 Community\data\commitlog\CommitLog-3-1392761510706.log
   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:120)
   at 
 org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:150)
   at 
 org.apache.cassandra.db.commitlog.CommitLogAllocator$4.run(CommitLogAllocator.java:217)
   at 
 org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at java.lang.Thread.run(Unknown Source)
 Caused by: java.nio.file.AccessDeniedException: C:\Program Files\DataStax 
 Community\data\commitlog\CommitLog-3-1392761510706.log
   at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
   at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source)
   at sun.nio.fs.AbstractFileSystemProvider.delete(Unknown Source)
   at java.nio.file.Files.delete(Unknown Source)
   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:116)
   ... 5 more
 (Attached in 2014-02-18-22-16.log is a larger excerpt from the cassandra.log.)
 In this particular case, I was trying to do 100 million inserts into two 
 tables in parallel, one with a single wide row and one with narrow rows, and 
 the error appeared after inserting 43,151,232 rows.  So it does take a while 
 to trip over this timing issue.  
 It may be aggravated by the size of the batches. This test was writing 10,000 
 rows to each table in a batch.  
 When I try switching the same test from using a logged batch to an unlogged 
 batch, and no such failure appears. So the issue could be related to the use 
 of large, logged batches, or it could be that unlogged batches just change 
 the probability of failure.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6738) java.lang.ClassCastException: org.apache.cassandra.db.composites.CompoundComposite cannot be cast to org.apache.cassandra.db.composites.CellName


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918174#comment-13918174
 ] 

Sylvain Lebresne commented on CASSANDRA-6738:
-

Slightly confused about those sstables. First because all of them (even the 
ones in the snapshot subdir) are on version jb, i.e. are already 2.1 sstables 
that don't need upgrading. But more importantly because those sstables have 
pretty clearly broken data (at least for the schema definition above). Mainly, 
there is a weird entry that do not correspond to the schema and in particular 
don't have the proper number of components in the cell name (which might be 
what trigger the exception above though I haven't been able to reproduce that 
one exactly, I do get other type of errors though since the sstable is clearly 
broken) which should never happen in a CQL3 table. It also doesn't appear that 
the cells are properly sorted in the sstable, suggesting something is really 
really wrong with the sstable.  

 java.lang.ClassCastException: 
 org.apache.cassandra.db.composites.CompoundComposite cannot be cast to 
 org.apache.cassandra.db.composites.CellName
 

 Key: CASSANDRA-6738
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6738
 Project: Cassandra
  Issue Type: Bug
Reporter: Mateusz Gajewski
Assignee: Sylvain Lebresne
 Fix For: 2.1 beta2

 Attachments: 6738.txt, user_attribs.tar.gz


 When using nodetool upgradesstables (2.0.4 - 2.1-beta) class cast exception 
 occurs:
 ERROR [CompactionExecutor:7] 2014-02-19 21:34:16,839 CassandraDaemon.java:165 
 - Exception in thread Thread[CompactionExecutor:7,1,main]
 java.lang.ClassCastException: 
 org.apache.cassandra.db.composites.CompoundComposite cannot be cast to 
 org.apache.cassandra.db.composites.CellName
   at 
 org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:86)
  ~[main/:na]
   at org.apache.cassandra.db.Cell$1.computeNext(Cell.java:75) ~[main/:na]
   at org.apache.cassandra.db.Cell$1.computeNext(Cell.java:64) ~[main/:na]
   at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
  ~[guava-16.0.jar:na]
   at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
 ~[guava-16.0.jar:na]
   at 
 org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:129)
  ~[main/:na]
   at 
 org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200)
  ~[main/:na]
   at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
  ~[guava-16.0.jar:na]
   at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
 ~[guava-16.0.jar:na]
   at 
 com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
 ~[guava-16.0.jar:na]
   at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
  ~[guava-16.0.jar:na]
   at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
 ~[guava-16.0.jar:na]
   at 
 org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:165)
  ~[main/:na]
   at 
 org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:110)
  ~[main/:na]
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:178) 
 ~[main/:na]
   at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:172)
  ~[main/:na]
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
  ~[main/:na]
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[main/:na]
   at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:67)
  ~[main/:na]
   at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:64)
  ~[main/:na]
   at 
 org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:262)
  ~[main/:na]
   at 
 org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:227)
  ~[main/:na]
   at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
 ~[na:1.7.0_45]
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  ~[na:1.7.0_45]
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_45]
   at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-6795) Forceful restart of C* during compaction on windows leads to exceptions on startup in scrubDataDirectories

Joshua McKenzie created CASSANDRA-6795:
--

 Summary: Forceful restart of C* during compaction on windows leads 
to exceptions on startup in scrubDataDirectories
 Key: CASSANDRA-6795
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6795
 Project: Cassandra
  Issue Type: Bug
 Environment: Windows 7, quad core, 8GB RAM, single Cassandra node, 
Cassandra 2.0.5 with leakdetect patch from CASSANDRA-6283
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
Priority: Minor


From comments on CASSANDRA-6736 (Bill Mitchell):

Trying to be polite, I started using Drain to shutdown Cassandra before 
rebooting the machine. In one case, this provoked numerous ThreadPoolExecutor 
has shutdown messages underneath the compactor:
INFO [RMI TCP Connection(2095)-127.0.0.1] 2014-02-24 08:34:23,743 
StorageService.java (line 947) DRAINING: starting drain process
INFO [RMI TCP Connection(2095)-127.0.0.1] 2014-02-24 08:34:23,783 
ThriftServer.java (line 141) Stop listening to thrift clients
INFO [RMI TCP Connection(2095)-127.0.0.1] 2014-02-24 08:34:24,980 Server.java 
(line 181) Stop listening for CQL clients
INFO [RMI TCP Connection(2095)-127.0.0.1] 2014-02-24 08:34:24,980 Gossiper.java 
(line 1251) Announcing shutdown
INFO [RMI TCP Connection(2095)-127.0.0.1] 2014-02-24 08:34:27,001 
MessagingService.java (line 665) Waiting for messaging service to quiesce
INFO [RMI TCP Connection(2095)-127.0.0.1] 2014-02-24 08:34:27,040 
ColumnFamilyStore.java (line 784) Enqueuing flush of 
Memtable-sr@1217138300(1825983/4411193 serialized/live bytes, 29946 ops)
INFO [RMI TCP Connection(2095)-127.0.0.1] 2014-02-24 08:34:27,040 
ColumnFamilyStore.java (line 784) Enqueuing flush of 
Memtable-etol@703118381(2963818/46129889 serialized/live bytes, 68926 ops)
INFO [FlushWriter:272] 2014-02-24 08:34:27,040 Memtable.java (line 333) Writing 
Memtable-sr@1217138300(1825983/4411193 serialized/live bytes, 29946 ops)
INFO [RMI TCP Connection(2095)-127.0.0.1] 2014-02-24 08:34:27,054 
ColumnFamilyStore.java (line 784) Enqueuing flush of 
Memtable-events@899982591(188/1880 serialized/live bytes, 7 ops)
INFO [RMI TCP Connection(2095)-127.0.0.1] 2014-02-24 08:34:27,075 
ColumnFamilyStore.java (line 784) Enqueuing flush of 
Memtable-events_timeline@1379706298(16/160 serialized/live bytes, 1 ops)
INFO [FlushWriter:273] 2014-02-24 08:34:27,075 Memtable.java (line 333) Writing 
Memtable-etol@703118381(2963818/46129889 serialized/live bytes, 68926 ops)
INFO [ACCEPT-localhost/127.0.0.1] 2014-02-24 08:34:27,144 MessagingService.java 
(line 875) MessagingService has terminated the accept() thread
INFO [FlushWriter:272] 2014-02-24 08:34:27,411 Memtable.java (line 373) 
Completed flushing C:\Program Files\DataStax 
Community\data\data\testdb_1393207231382\sr\testdb_1393207231382-sr-jb-473-Data.db
 (428854 bytes) for commitlog position ReplayPosition(segmentId=1393178353775, 
position=18771262)
INFO [FlushWriter:272] 2014-02-24 08:34:27,411 Memtable.java (line 333) Writing 
Memtable-events@899982591(188/1880 serialized/live bytes, 7 ops)
INFO [FlushWriter:273] 2014-02-24 08:34:27,932 Memtable.java (line 373) 
Completed flushing C:\Program Files\DataStax 
Community\data\data\testdb_1393207231382\etol\testdb_1393207231382-etol-jb-1563-Data.db
 (1012805 bytes) for commitlog position ReplayPosition(segmentId=1393178353775, 
position=18771262)
INFO [FlushWriter:273] 2014-02-24 08:34:27,933 Memtable.java (line 333) Writing 
Memtable-events_timeline@1379706298(16/160 serialized/live bytes, 1 ops)
INFO [FlushWriter:272] 2014-02-24 08:34:28,366 Memtable.java (line 373) 
Completed flushing C:\Program Files\DataStax 
Community\data\data\OpsCenter\events\OpsCenter-events-jb-32-Data.db (184 bytes) 
for commitlog position ReplayPosition(segmentId=1393178353775, 
position=18771262)
INFO [FlushWriter:273] 2014-02-24 08:34:28,456 Memtable.java (line 373) 
Completed flushing C:\Program Files\DataStax 
Community\data\data\OpsCenter\events_timeline\OpsCenter-events_timeline-jb-39-Data.db
 (47 bytes) for commitlog position ReplayPosition(segmentId=1393178353775, 
position=18771262)
INFO [RMI TCP Connection(2095)-127.0.0.1] 2014-02-24 08:34:28,457 
ColumnFamilyStore.java (line 784) Enqueuing flush of 
Memtable-compaction_history@814197203(1725/19675 serialized/live bytes, 45 ops)
INFO [FlushWriter:272] 2014-02-24 08:34:28,458 Memtable.java (line 333) Writing 
Memtable-compaction_history@814197203(1725/19675 serialized/live bytes, 45 ops)
INFO [RMI TCP Connection(2095)-127.0.0.1] 2014-02-24 08:34:28,458 
ColumnFamilyStore.java (line 784) Enqueuing flush of 
Memtable-sstable_activity@446592137(13500/207410 serialized/live bytes, 1442 
ops)
INFO [FlushWriter:273] 2014-02-24 08:34:28,458 Memtable.java (line 333) Writing 
Memtable-sstable_activity@446592137(13500/207410 serialized/live bytes, 1442 
ops)
INFO [FlushWriter:273] 2014-02-24

[jira] [Commented] (CASSANDRA-6736) Windows7 AccessDeniedException on commit log


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918188#comment-13918188
 ] 

Joshua McKenzie commented on CASSANDRA-6736:


Created CASSANDRA-6795 to track scrubDataDirectories exception.

 Windows7 AccessDeniedException on commit log 
 -

 Key: CASSANDRA-6736
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6736
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Windows 7, quad core, 8GB RAM, single Cassandra node, 
 Cassandra 2.0.5 with leakdetect patch from CASSANDRA-6283
Reporter: Bill Mitchell
Assignee: Joshua McKenzie
 Attachments: 2014-02-18-22-16.log


 Similar to the data file deletion of CASSANDRA-6283, under heavy load with 
 logged batches, I am seeing a problem where the Commit log cannot be deleted:
  ERROR [COMMIT-LOG-ALLOCATOR] 2014-02-18 22:15:58,252 CassandraDaemon.java 
 (line 192) Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
  FSWriteError in C:\Program Files\DataStax 
 Community\data\commitlog\CommitLog-3-1392761510706.log
   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:120)
   at 
 org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:150)
   at 
 org.apache.cassandra.db.commitlog.CommitLogAllocator$4.run(CommitLogAllocator.java:217)
   at 
 org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at java.lang.Thread.run(Unknown Source)
 Caused by: java.nio.file.AccessDeniedException: C:\Program Files\DataStax 
 Community\data\commitlog\CommitLog-3-1392761510706.log
   at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
   at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source)
   at sun.nio.fs.AbstractFileSystemProvider.delete(Unknown Source)
   at java.nio.file.Files.delete(Unknown Source)
   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:116)
   ... 5 more
 (Attached in 2014-02-18-22-16.log is a larger excerpt from the cassandra.log.)
 In this particular case, I was trying to do 100 million inserts into two 
 tables in parallel, one with a single wide row and one with narrow rows, and 
 the error appeared after inserting 43,151,232 rows.  So it does take a while 
 to trip over this timing issue.  
 It may be aggravated by the size of the batches. This test was writing 10,000 
 rows to each table in a batch.  
 When I try switching the same test from using a logged batch to an unlogged 
 batch, and no such failure appears. So the issue could be related to the use 
 of large, logged batches, or it could be that unlogged batches just change 
 the probability of failure.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918212#comment-13918212
 ] 

Viktor Kuzmin commented on CASSANDRA-6285:
--

I've tried to investigate problem with HsHaDistruptorServer, but with no luck. 
Telling the truth, I see no reason for that server to corrupt data.
Also HsHaDistruptor do not corrupt data in case useHeapBasedAllocation is 
turned on.
More over if you look at disruptor-thrift-server code - 
Message.reallocateDataBuffer and turn on heap based allocation only for 
dataBuffer then you will not see corruption.

 2.0 HSHA server introduces corrupt data
 ---

 Key: CASSANDRA-6285
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6285
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2
Reporter: David Sauer
Assignee: Pavel Yaskevich
Priority: Critical
 Fix For: 2.0.6

 Attachments: compaction_test.py


 After altering everything to LCS the table OpsCenter.rollups60 amd one other 
 none OpsCenter-Table got stuck with everything hanging around in L0.
 The compaction started and ran until the logs showed this:
 ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java 
 (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime]
 java.lang.RuntimeException: Last written key 
 DecoratedKey(1326283851463420237, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564)
  = current key DecoratedKey(954210699457429663, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f)
  writing into 
 /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
   at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:724)
 Moving back to STC worked to keep the compactions running.
 Especialy my own Table i would like to move to LCS.
 After a major compaction with STC the move to LCS fails with the same 
 Exception.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-4501) Gossip the ip and port used by the native protocol


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-4501:


Attachment: 4501.txt

Not that hard, here's a patch that adds it but doesn't do anything with it.  
I'll note though, that adding this in a minor means we'd need some kind of 
default for installations that don't have the parameter, so I guess to maintain 
the current behavior we'd have to fall back to using rpc_address :(  This patch 
doesn't do that.

 Gossip the ip and port used by the native protocol
 --

 Key: CASSANDRA-4501
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4501
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Brandon Williams
Priority: Minor
 Fix For: 2.1

 Attachments: 4501.txt


 Same as we gossip the rpc_address, we should add gossipping of the native 
 transport address (including the port). CASSANDRA-4480 has one reason why we 
 would want to do that.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-4501) Gossip the ip and port used by the native protocol


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-4501:


Attachment: (was: 4501.txt)

 Gossip the ip and port used by the native protocol
 --

 Key: CASSANDRA-4501
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4501
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Brandon Williams
Priority: Minor
 Fix For: 2.1

 Attachments: 4501.txt


 Same as we gossip the rpc_address, we should add gossipping of the native 
 transport address (including the port). CASSANDRA-4480 has one reason why we 
 would want to do that.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-4501) Gossip the ip and port used by the native protocol


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-4501:


Attachment: 4501.txt

 Gossip the ip and port used by the native protocol
 --

 Key: CASSANDRA-4501
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4501
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Brandon Williams
Priority: Minor
 Fix For: 2.1

 Attachments: 4501.txt


 Same as we gossip the rpc_address, we should add gossipping of the native 
 transport address (including the port). CASSANDRA-4480 has one reason why we 
 would want to do that.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-4501) Gossip the ip and port used by the native protocol


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918237#comment-13918237
 ] 

Brandon Williams commented on CASSANDRA-4501:
-

Oh, you changed this to 2.1, so nevermind.

 Gossip the ip and port used by the native protocol
 --

 Key: CASSANDRA-4501
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4501
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Brandon Williams
Priority: Minor
 Fix For: 2.1

 Attachments: 4501.txt


 Same as we gossip the rpc_address, we should add gossipping of the native 
 transport address (including the port). CASSANDRA-4480 has one reason why we 
 would want to do that.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-5483) Repair tracing

2014-03-03 Thread Lyuben Todorov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918251#comment-13918251
 ] 

Lyuben Todorov commented on CASSANDRA-5483:
---

Are the latest 3 patches supposed to be incrementally added onto 
{{trunk@4620823-5483-v02-0001-Trace-filtering-and-tracestate-propagation.patch}}
 and 
{{trunk@4620823-5483-v02-0002-Put-a-few-traces-parallel-to-the-repair-logging.patch}}?
 As in

{noformat}
1 - apply 
trunk@4620823-5483-v02-0001-Trace-filtering-and-tracestate-propagation.patch
2 - apply 
trunk@4620823-5483-v02-0002-Put-a-few-traces-parallel-to-the-repair-logging.patch
3 - apply one of the three latest patches (v3, v4 or v5)
{noformat}

v5 Does a lot of refactoring that I think is outside the scope of this ticket 
(but might be worth it's own ticket as the idea is good), so my vote is for v3, 
but I'm getting a NoSuchMethod exception, can you post a branch with all the 
patches added onto trunk (for v3)? 

The exception: 
{noformat}
java.lang.NoSuchMethodException: forceRepairAsync(java.lang.String, boolean, 
java.util.Collection, java.util.Collection, boolean, boolean, boolean, 
[Ljava.lang.String;)
at 
com.sun.jmx.mbeanserver.PerInterface.noSuchMethod(PerInterface.java:168)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:135)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at 
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at 
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at 
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at 
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at 
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at 
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}

bq. I am thinking of calling the new table something generic like 
system_traces.trace_logs. I also assume, that like system_traces.events

I'd say events is pretty generic, the new table should show that the traces 
aren't query related like in events. If we are going to add new tables to the 
trace CF it's worth thinking about refactoring events into something more 
specific and adding new tables with names that carry meaning. Another possible 
solution is to add a command field to system_traces.events where it can allow 
users to retrieve data about specific events, e.g. [~jbellis] WDYT? 

{noformat}
SELECT * FROM system_traces.events;
 session_id   | ... | thread| command
--+ ... +---+-
 09d48eb0-a2f1-11e3-9f04-7d9e3709bf93 | ... | Thrift:1  | REPAIR
 29084f90-a2f3-11e3-9f04-7d9e3709bf93 | ... | Thrift:1  | QUERY

(2 rows)

SELECT * FROM system_traces.events WHERE command='REPAIR';

 session_id   | ... | thread| command
--+ ... +---+-
 09d48eb0-a2f1-11e3-9f04-7d9e3709bf93 | ... | Thrift:1  | REPAIR

(1 rows)
{noformat}


bq. the rows in this table should expire, though perhaps not as fast as 24 
hours. 

+1, repairs can take a very long time so this should be configurable with the 
default perhaps being around 30 days, but with incremental repairs (in 2.1) it 
will end up logging a lot of data, still a better choice than users doing 
regular repairs missing out on information. 

bq. One last thing I wanted to ask is about the

[jira] [Comment Edited] (CASSANDRA-5483) Repair tracing

2014-03-03 Thread Lyuben Todorov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918251#comment-13918251
 ] 

Lyuben Todorov edited comment on CASSANDRA-5483 at 3/3/14 5:25 PM:
---

Are the latest 3 patches supposed to be incrementally added onto 
{{trunk@4620823-5483-v02-0001-Trace-filtering-and-tracestate-propagation.patch}}
 and 
{{trunk@4620823-5483-v02-0002-Put-a-few-traces-parallel-to-the-repair-logging.patch}}?
 As in

{noformat}
1 - apply 
trunk@4620823-5483-v02-0001-Trace-filtering-and-tracestate-propagation.patch
2 - apply 
trunk@4620823-5483-v02-0002-Put-a-few-traces-parallel-to-the-repair-logging.patch
3 - apply one of the three latest patches (v3, v4 or v5)
{noformat}

v5 Does a lot of refactoring that I think is outside the scope of this ticket 
(but might be worth it's own ticket as the idea is good), so my vote is for v3, 
but I'm getting a NoSuchMethod exception, can you post a branch with all the 
patches added onto trunk (for v3)? 

The exception: 
{noformat}
java.lang.NoSuchMethodException: forceRepairAsync(java.lang.String, boolean, 
java.util.Collection, java.util.Collection, boolean, boolean, boolean, 
[Ljava.lang.String;)
at 
com.sun.jmx.mbeanserver.PerInterface.noSuchMethod(PerInterface.java:168)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:135)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at 
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at 
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at 
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at 
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at 
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at 
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}

bq. I am thinking of calling the new table something generic like 
system_traces.trace_logs. I also assume, that like system_traces.events

I'd say events is pretty generic, the new table should show that the traces 
aren't query related like in events. If we are going to add new tables to the 
trace CF it's worth thinking about refactoring events into something more 
specific and adding new tables with names that carry meaning. Another possible 
solution is to add a command field to system_traces.events where it can allow 
users to retrieve data about specific events, e.g. [~jbellis] WDYT? 

{noformat}
SELECT * FROM system_traces.events;
 session_id   | ... | thread| command
--+ ... +---+-
 09d48eb0-a2f1-11e3-9f04-7d9e3709bf93 | ... | Thrift:1  | REPAIR
 29084f90-a2f3-11e3-9f04-7d9e3709bf93 | ... | Thrift:1  | QUERY

(2 rows)

SELECT * FROM system_traces.events WHERE command='REPAIR';

 session_id   | ... | thread| command
--+ ... +---+-
 09d48eb0-a2f1-11e3-9f04-7d9e3709bf93 | ... | Thrift:1  | REPAIR

(1 rows)
{noformat}


bq. the rows in this table should expire, though perhaps not as fast as 24 
hours. 

+1, repairs can take a very long time so this should be configurable with the 
default perhaps being around 90 days (but should be configurable), but with 
incremental repairs (in 2.1) it will end up logging a lot of data, still a 
better choice than users doing regular repairs missing

[jira] [Commented] (CASSANDRA-6745) Require specifying rows_per_partition_to_cache

2014-03-03 Thread Marcus Eriksson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918304#comment-13918304
 ] 

Marcus Eriksson commented on CASSANDRA-6745:


My reasoning was that it made no real sense to first have a boolean and then 
state how many rows, felt like i was repeating myself when doing it, but I can 
buy the argument that if we want to extend this in the future, having the bools 
might make sense

And the thrift part, sure

 Require specifying rows_per_partition_to_cache
 --

 Key: CASSANDRA-6745
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6745
 Project: Cassandra
  Issue Type: Bug
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
Priority: Trivial
 Fix For: 2.1 beta2


 We should require specifying rows_to_cache_per_partition for new tables or 
 newly ALTERed when row caching is enabled.
 Pre-upgrade should be grandfathered in as ALL to match existing semantics.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6756) Provide option to avoid loading orphan SSTables on startup

2014-03-03 Thread Vincent Mallet (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918313#comment-13918313
 ] 

Vincent Mallet commented on CASSANDRA-6756:
---

+1 on the lost+found idea.
Btw we're trying to analyze the source of some of these SSTables we're finding 
in some clusters and there seems to be other causes than failed repairs (in 
1.1) (OOM, problem with compaction, etc; still investigating). Having that 
option would make us sleep better at night.



 Provide option to avoid loading orphan SSTables on startup
 --

 Key: CASSANDRA-6756
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6756
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Vincent Mallet
 Fix For: 1.2.16


 When Cassandra starts up, it enumerates all SSTables on disk for a known 
 column family and proceeds to loading all of them, even those that were left 
 behind before the restart because of a problem of some sort. This can lead to 
 data gain (resurrected data) which is just as bad as data loss.
 The ask is to provide a yaml config option which would allow one to turn that 
 behavior off by default so a cassandra cluster would be immune to data gain 
 when nodes get restarted (at least with Leveled where Cassandra keeps track 
 of SSTables).
 This is sort of a follow-up to CASSANDRA-6503 (fixed in 1.2.14). We're just 
 extremely nervous that orphan SSTables could appear because of some other 
 potential problem somewhere else and cause zombie data on a random reboot. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-6796) StorageProxy may submit hint to itself (with an assert) for CL.Any

Viktor Kuzmin created CASSANDRA-6796:


 Summary: StorageProxy may submit hint to itself (with an assert) 
for CL.Any
 Key: CASSANDRA-6796
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6796
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Viktor Kuzmin
Priority: Minor


StorageProxy.mutate may  produce WriteTimoutException and with 
ConsistencyLevel.ANY it tries to submitHint. But submitHint function have 
assertation - we may not send hints to ourself. That may lead to exception (in 
case we're among natural endpoints) and hint will be not submitted to other 
endpoints.




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6745) Require specifying rows_per_partition_to_cache


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918337#comment-13918337
 ] 

Jonathan Ellis commented on CASSANDRA-6745:
---

I rather like the way Marcus did it.

 Require specifying rows_per_partition_to_cache
 --

 Key: CASSANDRA-6745
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6745
 Project: Cassandra
  Issue Type: Bug
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
Priority: Trivial
 Fix For: 2.1 beta2


 We should require specifying rows_to_cache_per_partition for new tables or 
 newly ALTERed when row caching is enabled.
 Pre-upgrade should be grandfathered in as ALL to match existing semantics.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6756) Provide option to avoid loading orphan SSTables on startup

[
https://issues.apache.org/jira/browse/CASSANDRA-6756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918338#comment-13918338
]

sankalp kohli commented on CASSANDRA-6756:
--

With this, we will also need an option to load all stables on startup. This
will be useful in cases where you intentionally drop stables in data directory.
Also will be useful during restore if system keyspace is not restored.

Provide option to avoid loading orphan SSTables on startup
--

Key: CASSANDRA-6756
URL: https://issues.apache.org/jira/browse/CASSANDRA-6756
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Vincent Mallet
Fix For: 1.2.16

When Cassandra starts up, it enumerates all SSTables on disk for a known
column family and proceeds to loading all of them, even those that were left
behind before the restart because of a problem of some sort. This can lead to
data gain (resurrected data) which is just as bad as data loss.
The ask is to provide a yaml config option which would allow one to turn that
behavior off by default so a cassandra cluster would be immune to data gain
when nodes get restarted (at least with Leveled where Cassandra keeps track
of SSTables).
This is sort of a follow-up to CASSANDRA-6503 (fixed in 1.2.14). We're just
extremely nervous that orphan SSTables could appear because of some other
potential problem somewhere else and cause zombie data on a random reboot.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918343#comment-13918343
 ] 

Pavel Yaskevich commented on CASSANDRA-6285:


[~ngrigoriev] and [~brandon.kearby] can you try setting useHeapBasedAllocation 
to true ? I'm fine with switch back to TThreadedSelectorServer if that helps.

 2.0 HSHA server introduces corrupt data
 ---

 Key: CASSANDRA-6285
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6285
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2
Reporter: David Sauer
Assignee: Pavel Yaskevich
Priority: Critical
 Fix For: 2.0.6

 Attachments: compaction_test.py


 After altering everything to LCS the table OpsCenter.rollups60 amd one other 
 none OpsCenter-Table got stuck with everything hanging around in L0.
 The compaction started and ran until the logs showed this:
 ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java 
 (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime]
 java.lang.RuntimeException: Last written key 
 DecoratedKey(1326283851463420237, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564)
  = current key DecoratedKey(954210699457429663, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f)
  writing into 
 /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
   at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:724)
 Moving back to STC worked to keep the compactions running.
 Especialy my own Table i would like to move to LCS.
 After a major compaction with STC the move to LCS fails with the same 
 Exception.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6788) Race condition silently kills thrift server

2014-03-03 Thread Christian Rolf (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian Rolf updated CASSANDRA-6788:
--

Attachment: 6788-v3.txt

True, given that the ThreadPoolExecutor.afterExecute is a noop, the exception 
should be rare. 

Here's an alternative solution...in the hope that you prefer overrides to 
factories and extra exception handling as much as I do :-)

 Race condition silently kills thrift server
 ---

 Key: CASSANDRA-6788
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6788
 Project: Cassandra
  Issue Type: Bug
Reporter: Christian Rolf
Assignee: Christian Rolf
 Attachments: 6788-v2.txt, 6788-v3.txt, race_patch.diff


 There's a race condition in CustomTThreadPoolServer that can cause the thrift 
 server to silently stop listening for connections. 
 It happens when the executor service throws a RejectedExecutionException, 
 which is not caught.
  
 Silent in the sense that OpsCenter doesn't notice any problem since JMX is 
 still running fine.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (CASSANDRA-6553) Benchmark counter improvements (counters++)

2014-03-03 Thread Ryan McGuire (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire reassigned CASSANDRA-6553:
---

Assignee: Russ Hatch  (was: Ryan McGuire)

 Benchmark counter improvements (counters++)
 ---

 Key: CASSANDRA-6553
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6553
 Project: Cassandra
  Issue Type: Test
Reporter: Ryan McGuire
Assignee: Russ Hatch
 Fix For: 2.1 beta2


 Benchmark the difference in performance between CASSANDRA-6504 and trunk.
 * Updating totally unrelated counters (different partitions)
 * Updating the same counters a lot (same cells in the same partition)
 * Different cells in the same few partitions (hot counter partition)
 benchmark: 
 https://github.com/apache/cassandra/tree/1218bcacba7edefaf56cf8440d0aea5794c89a1e
  (old counters)
 compared to: 
 https://github.com/apache/cassandra/tree/714c423360c36da2a2b365efaf9c5c4f623ed133
  (new counters)
 So far, the above changes should only affect the write path.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (CASSANDRA-6626) Create 2.0-2.1 counter upgrade dtests

2014-03-03 Thread Ryan McGuire (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire reassigned CASSANDRA-6626:
---

Assignee: Russ Hatch  (was: Ryan McGuire)

 Create 2.0-2.1 counter upgrade dtests
 --

 Key: CASSANDRA-6626
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6626
 Project: Cassandra
  Issue Type: Test
Reporter: Aleksey Yeschenko
Assignee: Russ Hatch
 Fix For: 2.1 beta2


 Create 2.0-2.1 counter upgrade dtests. Something more extensive, yet more 
 specific than 
 https://github.com/riptano/cassandra-dtest/blob/master/upgrade_through_versions_test.py



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6736) Windows7 AccessDeniedException on commit log


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918373#comment-13918373
 ] 

Joshua McKenzie commented on CASSANDRA-6736:


1st off - scratch what I said about the SSTableDeletingTask issue you saw above 
and 6283 - we're not seeing that stack there.  We're seeing tasks enqueued as 
expected but not getting file access errors.

2nd: Regarding the CommitLogSegment error: from the javadocs on 
MappedByteBuffer: A mapped byte buffer and the file mapping that it represents 
remain valid until the buffer itself is garbage-collected.

It looks like we've seen this type of situation before - from 
SSTableDeletingTask.java comments: Deleting sstables is tricky because the 
mmapping might not have been finalized yet, and delete will fail (on Windows) 
until it is (we only force the unmapping on SUN VMs). Additionally, we need to 
make sure to delete the data file first, so on restart the others will be 
recognized as GCable.

Our cleanup code in CommitLogSegment.java doesn't appear to take this into 
account:
{code:title=CommitLogSegment.java:148}
close();  // FileUtils.clean(buffer) in this method
if (deleteFile)
FileUtils.deleteWithConfirm(logFile);
{code}

Unless FileUtils.clean(buffer) actually works on Windows - which 
SSTableDeletingTask would imply does not - we may need to pursue deferring 
deletion attempts until after finalization on buffer.  Creating a 
DeferredDeletionTask that'll fire after gc's and retry to deal with this 
condition should be pretty straightforward.

Relevant bug from java bug db: 
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=4724038

 Windows7 AccessDeniedException on commit log 
 -

 Key: CASSANDRA-6736
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6736
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Windows 7, quad core, 8GB RAM, single Cassandra node, 
 Cassandra 2.0.5 with leakdetect patch from CASSANDRA-6283
Reporter: Bill Mitchell
Assignee: Joshua McKenzie
 Attachments: 2014-02-18-22-16.log


 Similar to the data file deletion of CASSANDRA-6283, under heavy load with 
 logged batches, I am seeing a problem where the Commit log cannot be deleted:
  ERROR [COMMIT-LOG-ALLOCATOR] 2014-02-18 22:15:58,252 CassandraDaemon.java 
 (line 192) Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
  FSWriteError in C:\Program Files\DataStax 
 Community\data\commitlog\CommitLog-3-1392761510706.log
   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:120)
   at 
 org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:150)
   at 
 org.apache.cassandra.db.commitlog.CommitLogAllocator$4.run(CommitLogAllocator.java:217)
   at 
 org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at java.lang.Thread.run(Unknown Source)
 Caused by: java.nio.file.AccessDeniedException: C:\Program Files\DataStax 
 Community\data\commitlog\CommitLog-3-1392761510706.log
   at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
   at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source)
   at sun.nio.fs.AbstractFileSystemProvider.delete(Unknown Source)
   at java.nio.file.Files.delete(Unknown Source)
   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:116)
   ... 5 more
 (Attached in 2014-02-18-22-16.log is a larger excerpt from the cassandra.log.)
 In this particular case, I was trying to do 100 million inserts into two 
 tables in parallel, one with a single wide row and one with narrow rows, and 
 the error appeared after inserting 43,151,232 rows.  So it does take a while 
 to trip over this timing issue.  
 It may be aggravated by the size of the batches. This test was writing 10,000 
 rows to each table in a batch.  
 When I try switching the same test from using a logged batch to an unlogged 
 batch, and no such failure appears. So the issue could be related to the use 
 of large, logged batches, or it could be that unlogged batches just change 
 the probability of failure.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6736) Windows7 AccessDeniedException on commit log


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918438#comment-13918438
 ] 

Jonathan Ellis commented on CASSANDRA-6736:
---

I think that comment was saying two related things:

- We need to unmap before deleting on Windows
- We only have code to unmap for Sun (Oracle) VM, so all bets are off for e.g. 
IBM VM

I have no reason to believe FileUtils.clean does not work on Windows.

 Windows7 AccessDeniedException on commit log 
 -

 Key: CASSANDRA-6736
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6736
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Windows 7, quad core, 8GB RAM, single Cassandra node, 
 Cassandra 2.0.5 with leakdetect patch from CASSANDRA-6283
Reporter: Bill Mitchell
Assignee: Joshua McKenzie
 Attachments: 2014-02-18-22-16.log


 Similar to the data file deletion of CASSANDRA-6283, under heavy load with 
 logged batches, I am seeing a problem where the Commit log cannot be deleted:
  ERROR [COMMIT-LOG-ALLOCATOR] 2014-02-18 22:15:58,252 CassandraDaemon.java 
 (line 192) Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
  FSWriteError in C:\Program Files\DataStax 
 Community\data\commitlog\CommitLog-3-1392761510706.log
   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:120)
   at 
 org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:150)
   at 
 org.apache.cassandra.db.commitlog.CommitLogAllocator$4.run(CommitLogAllocator.java:217)
   at 
 org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at java.lang.Thread.run(Unknown Source)
 Caused by: java.nio.file.AccessDeniedException: C:\Program Files\DataStax 
 Community\data\commitlog\CommitLog-3-1392761510706.log
   at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
   at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source)
   at sun.nio.fs.AbstractFileSystemProvider.delete(Unknown Source)
   at java.nio.file.Files.delete(Unknown Source)
   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:116)
   ... 5 more
 (Attached in 2014-02-18-22-16.log is a larger excerpt from the cassandra.log.)
 In this particular case, I was trying to do 100 million inserts into two 
 tables in parallel, one with a single wide row and one with narrow rows, and 
 the error appeared after inserting 43,151,232 rows.  So it does take a while 
 to trip over this timing issue.  
 It may be aggravated by the size of the batches. This test was writing 10,000 
 rows to each table in a batch.  
 When I try switching the same test from using a logged batch to an unlogged 
 batch, and no such failure appears. So the issue could be related to the use 
 of large, logged batches, or it could be that unlogged batches just change 
 the probability of failure.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (CASSANDRA-6761) A positive value of row_cache_save_period is needed to enable row cache


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sankalp kohli resolved CASSANDRA-6761.
--

Resolution: Cannot Reproduce

Please reopen it if this is an issue. 

 A positive value of row_cache_save_period is needed to enable row cache
 ---

 Key: CASSANDRA-6761
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6761
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Supencheck
Assignee: sankalp kohli
Priority: Minor

 When the setting for row_cache_save_period is 0, the value specified for 
 row_cache_size_in_mb is not utilized.
 The cassandra version with which this behavior has been observed is:
 adminuser@Ubuntu1:~$ nodetool version
 ReleaseVersion: 1.2.13.2
 To replicate
 1) Stop dse;
 2) Edit the cassandra.yaml file, setting the value of the 
 row_cache_size_in_mb to be a nonzero value;
 3) Start dse;
 4) Run the 'nodetool info' command and observe the following line in the 
 output:
 Row Cache : size 0 (bytes), capacity 0 (bytes), 0 hits, 0 requests, NaN 
 recent hit rate, 0 save period in seconds.
 The expected behavior is that the value of the capacity parameter in the Row 
 Cache row of the above nodetool output would report the setting for 
 row_cache_size_in_mb.  If the value of row_cache_save_period is set to a 
 positive value, the nodetool output correctly reports the capacity as the 
 value for the row_cache_size_in_mb parameter.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6475) Control nodetool history logging directory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918454#comment-13918454
 ] 

sankalp kohli commented on CASSANDRA-6475:
--

[~brandon.williams] Can you review it? 

 Control nodetool history logging directory
 --

 Key: CASSANDRA-6475
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6475
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Reporter: Brian Nixon
Assignee: sankalp kohli
Priority: Trivial
  Labels: lhf
 Attachments: trunk-6475.diff


 Nodetool history is logged to a directory based on the current user home. 
 This leads to splintering of history with more than one user and, in one 
 case, a failure to run any nodetool commands as the user did not have write 
 access to their home directory.
 Suggested fix is to make the base directory for the history logging (both 
 nodetool and cli) configurable. A way to disable the logging of these tools 
 would also help.
 Reference:
 https://issues.apache.org/jira/browse/CASSANDRA-5823



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6475) Control nodetool history logging directory


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6475:


Reviewer: Brandon Williams

 Control nodetool history logging directory
 --

 Key: CASSANDRA-6475
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6475
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Reporter: Brian Nixon
Assignee: sankalp kohli
Priority: Trivial
  Labels: lhf
 Attachments: trunk-6475.diff


 Nodetool history is logged to a directory based on the current user home. 
 This leads to splintering of history with more than one user and, in one 
 case, a failure to run any nodetool commands as the user did not have write 
 access to their home directory.
 Suggested fix is to make the base directory for the history logging (both 
 nodetool and cli) configurable. A way to disable the logging of these tools 
 would also help.
 Reference:
 https://issues.apache.org/jira/browse/CASSANDRA-5823



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data

2014-03-03 Thread Nikolai Grigoriev (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918463#comment-13918463
 ] 

Nikolai Grigoriev commented on CASSANDRA-6285:
--

[~xedin]

That seems to be a parameter of the Thrift server...How do I control this 
parameter? Or I should just disable JNA?

 2.0 HSHA server introduces corrupt data
 ---

 Key: CASSANDRA-6285
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6285
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2
Reporter: David Sauer
Assignee: Pavel Yaskevich
Priority: Critical
 Fix For: 2.0.6

 Attachments: compaction_test.py


 After altering everything to LCS the table OpsCenter.rollups60 amd one other 
 none OpsCenter-Table got stuck with everything hanging around in L0.
 The compaction started and ran until the logs showed this:
 ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java 
 (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime]
 java.lang.RuntimeException: Last written key 
 DecoratedKey(1326283851463420237, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564)
  = current key DecoratedKey(954210699457429663, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f)
  writing into 
 /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
   at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:724)
 Moving back to STC worked to keep the compactions running.
 Especialy my own Table i would like to move to LCS.
 After a major compaction with STC the move to LCS fails with the same 
 Exception.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6475) Control nodetool history logging directory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918478#comment-13918478
 ] 

Brandon Williams commented on CASSANDRA-6475:
-

USER_HOME as an env var seems odd to me, shouldn't we just respect HOME?

 Control nodetool history logging directory
 --

 Key: CASSANDRA-6475
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6475
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Reporter: Brian Nixon
Assignee: sankalp kohli
Priority: Trivial
  Labels: lhf
 Attachments: trunk-6475.diff


 Nodetool history is logged to a directory based on the current user home. 
 This leads to splintering of history with more than one user and, in one 
 case, a failure to run any nodetool commands as the user did not have write 
 access to their home directory.
 Suggested fix is to make the base directory for the history logging (both 
 nodetool and cli) configurable. A way to disable the logging of these tools 
 would also help.
 Reference:
 https://issues.apache.org/jira/browse/CASSANDRA-5823



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6475) Control nodetool history logging directory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918491#comment-13918491
 ] 

Brandon Williams commented on CASSANDRA-6475:
-

Actually, os.path.expanduser('~') should already expand to HOME

 Control nodetool history logging directory
 --

 Key: CASSANDRA-6475
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6475
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Reporter: Brian Nixon
Assignee: sankalp kohli
Priority: Trivial
  Labels: lhf
 Attachments: trunk-6475.diff


 Nodetool history is logged to a directory based on the current user home. 
 This leads to splintering of history with more than one user and, in one 
 case, a failure to run any nodetool commands as the user did not have write 
 access to their home directory.
 Suggested fix is to make the base directory for the history logging (both 
 nodetool and cli) configurable. A way to disable the logging of these tools 
 would also help.
 Reference:
 https://issues.apache.org/jira/browse/CASSANDRA-5823



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918496#comment-13918496
 ] 

Pavel Yaskevich commented on CASSANDRA-6285:


You can do it via JMX or disable JNA, I can also make a patch with would set it 
explicitly in Cassandra code.

 2.0 HSHA server introduces corrupt data
 ---

 Key: CASSANDRA-6285
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6285
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2
Reporter: David Sauer
Assignee: Pavel Yaskevich
Priority: Critical
 Fix For: 2.0.6

 Attachments: compaction_test.py


 After altering everything to LCS the table OpsCenter.rollups60 amd one other 
 none OpsCenter-Table got stuck with everything hanging around in L0.
 The compaction started and ran until the logs showed this:
 ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java 
 (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime]
 java.lang.RuntimeException: Last written key 
 DecoratedKey(1326283851463420237, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564)
  = current key DecoratedKey(954210699457429663, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f)
  writing into 
 /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
   at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:724)
 Moving back to STC worked to keep the compactions running.
 Especialy my own Table i would like to move to LCS.
 After a major compaction with STC the move to LCS fails with the same 
 Exception.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6475) Control nodetool history logging directory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918503#comment-13918503
 ] 

sankalp kohli commented on CASSANDRA-6475:
--

The environment variable is user.home and not USER_HOME.
We use user.home in node tool as well. 

 Control nodetool history logging directory
 --

 Key: CASSANDRA-6475
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6475
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Reporter: Brian Nixon
Assignee: sankalp kohli
Priority: Trivial
  Labels: lhf
 Attachments: trunk-6475.diff


 Nodetool history is logged to a directory based on the current user home. 
 This leads to splintering of history with more than one user and, in one 
 case, a failure to run any nodetool commands as the user did not have write 
 access to their home directory.
 Suggested fix is to make the base directory for the history logging (both 
 nodetool and cli) configurable. A way to disable the logging of these tools 
 would also help.
 Reference:
 https://issues.apache.org/jira/browse/CASSANDRA-5823



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-5201) Cassandra/Hadoop does not support current Hadoop releases

2014-03-03 Thread Henning Kropp (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918536#comment-13918536
 ] 

Henning Kropp commented on CASSANDRA-5201:
--

I also was able to make this patch work with HDP 2.0. Nevertheless before this 
ticket is closed I would like to make 2 suggestions:
1. Use maven classifiers for deployment. If this is used in a hadoop2 project 
you don't want hadoop1 dependencies and vice versa. This approach is used by 
Parquet, Avro and EB. For detailed discussion please read 
[here|https://github.com/Parquet/parquet-mr/pull/32].
2. Change {{hadoop-core}} to {{hadoop-client}} dependency. {{hadoop-client}} is 
the preferred dependency. Read 
[here|http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH-Version-and-Packaging-Information/cdhvd_topic_8_1.html]
 and the link above.

 Cassandra/Hadoop does not support current Hadoop releases
 -

 Key: CASSANDRA-5201
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5201
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.0
Reporter: Brian Jeltema
Assignee: Benjamin Coverston
 Fix For: 2.0.6

 Attachments: 5201_a.txt, hadoopCompat.patch, 
 hadoopcompat-trunk.patch, progressable-fix.patch


 Using Hadoop 0.22.0 with Cassandra results in the stack trace below.
 It appears that version 0.21+ changed org.apache.hadoop.mapreduce.JobContext
 from a class to an interface.
 Exception in thread main java.lang.IncompatibleClassChangeError: Found 
 interface org.apache.hadoop.mapreduce.JobContext, but class was expected
   at 
 org.apache.cassandra.hadoop.ColumnFamilyInputFormat.getSplits(ColumnFamilyInputFormat.java:103)
   at 
 org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:445)
   at 
 org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:462)
   at 
 org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:357)
   at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1045)
   at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1042)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:415)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1153)
   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1042)
   at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1062)
   at MyHadoopApp.run(MyHadoopApp.java:163)
   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
   at MyHadoopApp.main(MyHadoopApp.java:82)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:192)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6475) Control nodetool history logging directory

2014-03-03 Thread Mikhail Stepura (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918555#comment-13918555
 ] 

Mikhail Stepura commented on CASSANDRA-6475:


I believe pyhton's {{os.path.expanduser('~')}} is the same as Java's 
{{System.getProperty(user.home)}}. And {{user.home}} is a Java System 
property (not an env. variable) which expands to a real env. variable.


 Control nodetool history logging directory
 --

 Key: CASSANDRA-6475
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6475
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Reporter: Brian Nixon
Assignee: sankalp kohli
Priority: Trivial
  Labels: lhf
 Attachments: trunk-6475.diff


 Nodetool history is logged to a directory based on the current user home. 
 This leads to splintering of history with more than one user and, in one 
 case, a failure to run any nodetool commands as the user did not have write 
 access to their home directory.
 Suggested fix is to make the base directory for the history logging (both 
 nodetool and cli) configurable. A way to disable the logging of these tools 
 would also help.
 Reference:
 https://issues.apache.org/jira/browse/CASSANDRA-5823



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6736) Windows7 AccessDeniedException on commit log


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918580#comment-13918580
 ] 

Joshua McKenzie commented on CASSANDRA-6736:


Stress-testing the FileUtils MappedByteBuffer and RAF.close() on Windows looks 
clean.  I agree that there's no indication that's the issue - thanks for 
clarifying the Windows + non-Sun JVM nature of that earlier comment.

 Windows7 AccessDeniedException on commit log 
 -

 Key: CASSANDRA-6736
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6736
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Windows 7, quad core, 8GB RAM, single Cassandra node, 
 Cassandra 2.0.5 with leakdetect patch from CASSANDRA-6283
Reporter: Bill Mitchell
Assignee: Joshua McKenzie
 Attachments: 2014-02-18-22-16.log


 Similar to the data file deletion of CASSANDRA-6283, under heavy load with 
 logged batches, I am seeing a problem where the Commit log cannot be deleted:
  ERROR [COMMIT-LOG-ALLOCATOR] 2014-02-18 22:15:58,252 CassandraDaemon.java 
 (line 192) Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
  FSWriteError in C:\Program Files\DataStax 
 Community\data\commitlog\CommitLog-3-1392761510706.log
   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:120)
   at 
 org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:150)
   at 
 org.apache.cassandra.db.commitlog.CommitLogAllocator$4.run(CommitLogAllocator.java:217)
   at 
 org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at java.lang.Thread.run(Unknown Source)
 Caused by: java.nio.file.AccessDeniedException: C:\Program Files\DataStax 
 Community\data\commitlog\CommitLog-3-1392761510706.log
   at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
   at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source)
   at sun.nio.fs.AbstractFileSystemProvider.delete(Unknown Source)
   at java.nio.file.Files.delete(Unknown Source)
   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:116)
   ... 5 more
 (Attached in 2014-02-18-22-16.log is a larger excerpt from the cassandra.log.)
 In this particular case, I was trying to do 100 million inserts into two 
 tables in parallel, one with a single wide row and one with narrow rows, and 
 the error appeared after inserting 43,151,232 rows.  So it does take a while 
 to trip over this timing issue.  
 It may be aggravated by the size of the batches. This test was writing 10,000 
 rows to each table in a batch.  
 When I try switching the same test from using a logged batch to an unlogged 
 batch, and no such failure appears. So the issue could be related to the use 
 of large, logged batches, or it could be that unlogged batches just change 
 the probability of failure.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6689) Partially Off Heap Memtables


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918701#comment-13918701
 ] 

Pavel Yaskevich commented on CASSANDRA-6689:


I read through most of the code in offheap1b skipping queue implementations add 
there and I have couple of questions:

I see this pattern in the code (e.g. {RangeSlice, Read}VerbHandler.java) where 
Referrer is allocated try { operation(); referrer = null } finally { 
referrer.setDone() if (referrer != null); } where documentation says that that 
setDone() should be called when we are done with the referrer, can you 
elaborate why do we need to explicitly set it to null?

Pool.java - can we schedule cleaners to a static thread pool (or something 
similar) instead of creating a thread per cleaner? With would add more control 
in the situation when we flush memtables in short intervals. Related question, 
why can't we have per-cf (or global) pool instead of creating it per memtable, 
might be possible to remove allocator groups if we do that?

Also I just want to throw an idea here maybe there is something useful in it - 
as we have fixed number of stages and threads in them and don't normally share 
much between them, maybe it would be possible to use that as an advantage for 
allocator gc? What I mean is, it might be possible to track life-time of the 
allocation not by how buffer was allocated but where it was allocated, more 
like what RCU...

 Partially Off Heap Memtables
 

 Key: CASSANDRA-6689
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 2.1 beta2


 Move the contents of ByteBuffers off-heap for records written to a memtable.
 (See comments for details)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-6689) Partially Off Heap Memtables


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918701#comment-13918701
 ] 

Pavel Yaskevich edited comment on CASSANDRA-6689 at 3/3/14 11:15 PM:
-

I read through most of the code in offheap1b skipping queue implementations add 
there and I have couple of questions:

I see this pattern in the code (e.g. (RangeSlice, Read)VerbHandler.java) where 
{noformat}
r = Referrer(); 
try { operation(); r = null } 
finally { if (r != null) r.setDone(); } 
{noformat}
 where documentation says that that setDone() should be called when we are done 
with the referrer, can you elaborate why do we need to explicitly set it to 
null?

Pool.java - can we schedule cleaners to a static thread pool (or something 
similar) instead of creating a thread per cleaner? With would add more control 
in the situation when we flush memtables in short intervals. Related question, 
why can't we have per-cf (or global) pool instead of creating it per memtable, 
might be possible to remove allocator groups if we do that?

Also I just want to throw an idea here maybe there is something useful in it - 
as we have fixed number of stages and threads in them and don't normally share 
much between them, maybe it would be possible to use that as an advantage for 
allocator gc? What I mean is, it might be possible to track life-time of the 
allocation not by how buffer was allocated but where it was allocated, more 
like what RCU...


was (Author: xedin):
I read through most of the code in offheap1b skipping queue implementations add 
there and I have couple of questions:

I see this pattern in the code (e.g. {RangeSlice, Read}VerbHandler.java) where 
Referrer is allocated try { operation(); referrer = null } finally { 
referrer.setDone() if (referrer != null); } where documentation says that that 
setDone() should be called when we are done with the referrer, can you 
elaborate why do we need to explicitly set it to null?

Pool.java - can we schedule cleaners to a static thread pool (or something 
similar) instead of creating a thread per cleaner? With would add more control 
in the situation when we flush memtables in short intervals. Related question, 
why can't we have per-cf (or global) pool instead of creating it per memtable, 
might be possible to remove allocator groups if we do that?

Also I just want to throw an idea here maybe there is something useful in it - 
as we have fixed number of stages and threads in them and don't normally share 
much between them, maybe it would be possible to use that as an advantage for 
allocator gc? What I mean is, it might be possible to track life-time of the 
allocation not by how buffer was allocated but where it was allocated, more 
like what RCU...

 Partially Off Heap Memtables
 

 Key: CASSANDRA-6689
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 2.1 beta2


 Move the contents of ByteBuffers off-heap for records written to a memtable.
 (See comments for details)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6689) Partially Off Heap Memtables

2014-03-03 Thread Benedict (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918726#comment-13918726
]

Benedict commented on CASSANDRA-6689:
-

bq. can you elaborate why do we need to explicitly set it to null?

I do this in places where I'm handing off control of the referrer to another
thread/context, so we only call setDone() in the case of an error before we did
this. Admittedly, looking at these places now, it looks like replacing the
finally block with a catch (Throwable t) block would be fine, and given we can
do untyped rethrow there isn't anything lost by doing this. Probably an old
habit from Java 6 to do it the way I did.

bq. With would add more control in the situation when we flush memtables in
short intervals.

If I understand you correctly, you mean to deal with the situation where we
want to flush memtables quick enough that the single cleaner thread may become
the bottleneck? In which case, I don't think this should ever be a problem: the
work done by the cleaner itself is very minimal unless it's got some GC to do,
in which case it is potentially non-trivial, but still unlikely to be a
bottleneck. There might be a case for handing off the GC candidate selection to
another thread, except that we expect it to certainly be quicker than any
flush, and we want to wait for it to complete before deciding if flushing is
actually worthwhile, so I'm not sure it buys us much. Maybe we could do this
and continue to check the memory state isn't worsening, and if it is just force
a flush without waiting for the GC, but it probably is too unlikely to be of
benefit to be worth the extra complexity. That is, unless you meant some other
benefit I missed?

bq. Related question, why can't we have per-cf (or global) pool instead of
creating it per memtable, might be possible to remove allocator groups if we do
that?

The pool _is_ global; a per-cf pool would have the issue of poor sharing of
resources, though, no? I would probably like to see us move to allocators
surviving across memtable replacement, so that we could then flatten allocator
group with allocator, but I'd prefer to save that for a later date.

bq. Also I just want to throw an idea here maybe there is something useful in
it - as we have fixed number of stages and threads in them and don't normally
share much between them, maybe it would be possible to use that as an advantage
for allocator gc? What I mean is, it might be possible to track life-time of
the allocation not by how buffer was allocated but where it was allocated, more
like what RCU...

Not really sure what you're suggesting here, could you elaborate?

Partially Off Heap Memtables

Key: CASSANDRA-6689
URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Benedict
Assignee: Benedict
Fix For: 2.1 beta2

Move the contents of ByteBuffers off-heap for records written to a memtable.
(See comments for details)

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-5201) Cassandra/Hadoop does not support current Hadoop releases

2014-03-03 Thread Benjamin Coverston (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918725#comment-13918725
 ] 

Benjamin Coverston commented on CASSANDRA-5201:
---

[~hkropp] I agree that we need to do the second change, but I don't see a need 
to add deployment classifiers. Because we're using Hadoop-Compat (from EB) a 
single binary will work for both.

There's a niggle right now with Progressable that I'm working on, but a single 
binary will work for new and legacy deployments.

 Cassandra/Hadoop does not support current Hadoop releases
 -

 Key: CASSANDRA-5201
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5201
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.0
Reporter: Brian Jeltema
Assignee: Benjamin Coverston
 Fix For: 2.0.6

 Attachments: 5201_a.txt, hadoopCompat.patch, 
 hadoopcompat-trunk.patch, progressable-fix.patch


 Using Hadoop 0.22.0 with Cassandra results in the stack trace below.
 It appears that version 0.21+ changed org.apache.hadoop.mapreduce.JobContext
 from a class to an interface.
 Exception in thread main java.lang.IncompatibleClassChangeError: Found 
 interface org.apache.hadoop.mapreduce.JobContext, but class was expected
   at 
 org.apache.cassandra.hadoop.ColumnFamilyInputFormat.getSplits(ColumnFamilyInputFormat.java:103)
   at 
 org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:445)
   at 
 org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:462)
   at 
 org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:357)
   at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1045)
   at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1042)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:415)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1153)
   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1042)
   at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1062)
   at MyHadoopApp.run(MyHadoopApp.java:163)
   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
   at MyHadoopApp.main(MyHadoopApp.java:82)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:192)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-6689) Partially Off Heap Memtables

2014-03-03 Thread Benedict (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918726#comment-13918726
]

Benedict edited comment on CASSANDRA-6689 at 3/3/14 11:35 PM:
--

bq. can you elaborate why do we need to explicitly set it to null?

bq. With would add more control in the situation when we flush memtables in
short intervals.

bq. Related question, why can't we have per-cf (or global) pool instead of
creating it per memtable, might be possible to remove allocator groups if we do
that?

Not really sure what you're suggesting here, could you elaborate?

was (Author: benedict):
bq. can you elaborate why do we need to explicitly set it to null?

bq. With would add more control in the situation when we flush memtables in
short intervals.

bq. Related question, why can't we have per-cf (or global) pool instead of
creating it per memtable, might be possible to remove allocator groups if we do
that?

bq. Also I just want to throw an idea here maybe there is something useful in
it - as we

[jira] [Commented] (CASSANDRA-6786) cassandra-stress failing in 2.1-beta1+ with current java-driver

2014-03-03 Thread Mikhail Stepura (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918761#comment-13918761
 ] 

Mikhail Stepura commented on CASSANDRA-6786:


Duplicate of CASSANDRA-6777?

 cassandra-stress failing in 2.1-beta1+ with current java-driver
 ---

 Key: CASSANDRA-6786
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6786
 Project: Cassandra
  Issue Type: Bug
Reporter: Michael Shuler
 Fix For: 2.1 beta2

 Attachments: cassandra-2.1_pre-beta2.log


 The c*-beta1 tar and deb packages went out without a working 
 cassandra-stress, which should be fixed in the packaging (CASSANDRA-6762). I 
 built the current java-driver and dropped it into cassandra-2.1 branch to 
 verify the c* version check changes work OK, and I get /0 errors running 
 stress. (stress with the 2.0.0-rc3 driver currently in tools/lib/ works if 
 'ant -Dbase.version=2.1.0 ...' is passed at c* build time)
 java-driver - commit e4453ae
 cassandra-2.1 - commit dcca996



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6475) Control nodetool history logging directory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918764#comment-13918764
 ] 

sankalp kohli commented on CASSANDRA-6475:
--

With my change you can run cqlsh like this. env user.home=/tmp ./cqlsh


 Control nodetool history logging directory
 --

 Key: CASSANDRA-6475
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6475
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Reporter: Brian Nixon
Assignee: sankalp kohli
Priority: Trivial
  Labels: lhf
 Attachments: trunk-6475.diff


 Nodetool history is logged to a directory based on the current user home. 
 This leads to splintering of history with more than one user and, in one 
 case, a failure to run any nodetool commands as the user did not have write 
 access to their home directory.
 Suggested fix is to make the base directory for the history logging (both 
 nodetool and cli) configurable. A way to disable the logging of these tools 
 would also help.
 Reference:
 https://issues.apache.org/jira/browse/CASSANDRA-5823



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (CASSANDRA-6786) cassandra-stress failing in 2.1-beta1+ with current java-driver

2014-03-03 Thread Michael Shuler (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler resolved CASSANDRA-6786.
---

Resolution: Duplicate

 cassandra-stress failing in 2.1-beta1+ with current java-driver
 ---

 Key: CASSANDRA-6786
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6786
 Project: Cassandra
  Issue Type: Bug
Reporter: Michael Shuler
 Fix For: 2.1 beta2

 Attachments: cassandra-2.1_pre-beta2.log


 The c*-beta1 tar and deb packages went out without a working 
 cassandra-stress, which should be fixed in the packaging (CASSANDRA-6762). I 
 built the current java-driver and dropped it into cassandra-2.1 branch to 
 verify the c* version check changes work OK, and I get /0 errors running 
 stress. (stress with the 2.0.0-rc3 driver currently in tools/lib/ works if 
 'ant -Dbase.version=2.1.0 ...' is passed at c* build time)
 java-driver - commit e4453ae
 cassandra-2.1 - commit dcca996



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6475) Control nodetool history logging directory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918776#comment-13918776
 ] 

Brandon Williams commented on CASSANDRA-6475:
-

You can achieve the same effect already with {{HOME=/tmp bin/cqlsh}}

 Control nodetool history logging directory
 --

 Key: CASSANDRA-6475
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6475
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Reporter: Brian Nixon
Assignee: sankalp kohli
Priority: Trivial
  Labels: lhf
 Attachments: trunk-6475.diff


 Nodetool history is logged to a directory based on the current user home. 
 This leads to splintering of history with more than one user and, in one 
 case, a failure to run any nodetool commands as the user did not have write 
 access to their home directory.
 Suggested fix is to make the base directory for the history logging (both 
 nodetool and cli) configurable. A way to disable the logging of these tools 
 would also help.
 Reference:
 https://issues.apache.org/jira/browse/CASSANDRA-5823



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6689) Partially Off Heap Memtables


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918785#comment-13918785
 ] 

Pavel Yaskevich commented on CASSANDRA-6689:


bq. If I understand you correctly, you mean to deal with the situation where we 
want to flush memtables quick enough that the single cleaner thread may become 
the bottleneck? In which case, I don't think this should ever be a problem: the 
work done by the cleaner itself is very minimal unless it's got some GC to do, 
in which case it is potentially non-trivial, but still unlikely to be a 
bottleneck. There might be a case for handing off the GC candidate selection to 
another thread, except that we expect it to certainly be quicker than any 
flush, and we want to wait for it to complete before deciding if flushing is 
actually worthwhile, so I'm not sure it buys us much. Maybe we could do this 
and continue to check the memory state isn't worsening, and if it is just force 
a flush without waiting for the GC, but it probably is too unlikely to be of 
benefit to be worth the extra complexity. That is, unless you meant some other 
benefit I missed?

I can image the situation when we frequently switch memtables which, in current 
code, starts a new thread, so I wonder how quickly we would be able to see a 
slowdown affect caused by that + potentially OOM on stack space...

bq. The pool is global; a per-cf pool would have the issue of poor sharing of 
resources, though, no? I would probably like to see us move to allocators 
surviving across memtable replacement, so that we could then flatten allocator 
group with allocator, but I'd prefer to save that for a later date.

I'm thinking maybe it would lessen internal complexity if we do (sort-of) arena 
allocation inside instead of having groups and sub pools?

bq. Not really sure what you're suggesting here, could you elaborate?

It's just an idea so people can just ignore it but what I meant there is maybe 
it would be a better first step for us to consider making all of the operations 
that we do more self-contained so we can track exactly where it started and 
ends (by operation I mean all types of reads, writes), which could potentially 
make allocator job easier as we would be able to container  per operation 
memory...

 Partially Off Heap Memtables
 

 Key: CASSANDRA-6689
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 2.1 beta2


 Move the contents of ByteBuffers off-heap for records written to a memtable.
 (See comments for details)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-6689) Partially Off Heap Memtables


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918785#comment-13918785
 ] 

Pavel Yaskevich edited comment on CASSANDRA-6689 at 3/4/14 12:17 AM:
-

bq. If I understand you correctly, you mean to deal with the situation where we 
want to flush memtables quick enough that the single cleaner thread may become 
the bottleneck? In which case, I don't think this should ever be a problem: the 
work done by the cleaner itself is very minimal unless it's got some GC to do, 
in which case it is potentially non-trivial, but still unlikely to be a 
bottleneck. There might be a case for handing off the GC candidate selection to 
another thread, except that we expect it to certainly be quicker than any 
flush, and we want to wait for it to complete before deciding if flushing is 
actually worthwhile, so I'm not sure it buys us much. Maybe we could do this 
and continue to check the memory state isn't worsening, and if it is just force 
a flush without waiting for the GC, but it probably is too unlikely to be of 
benefit to be worth the extra complexity. That is, unless you meant some other 
benefit I missed?

I can image the situation when we frequently switch memtables which, in current 
code, starts a new thread, so I wonder how quickly we would be able to see a 
slowdown affect caused by that + potentially OOM on stack space...

bq. The pool is global; a per-cf pool would have the issue of poor sharing of 
resources, though, no? I would probably like to see us move to allocators 
surviving across memtable replacement, so that we could then flatten allocator 
group with allocator, but I'd prefer to save that for a later date.

I'm thinking maybe it would lessen internal complexity if we do (sort-of) arena 
allocation inside instead of having groups and sub pools?

bq. Not really sure what you're suggesting here, could you elaborate?

It's an idea so people can just ignore it but what I meant there is maybe it 
would be a better first step for us to consider making all of the operations 
that we do more self-contained so we can track exactly where it started and 
ends (by operation I mean all types of reads, writes), which could potentially 
make allocator job easier as we would be able to allocate memory per operation 
and get rid of RefAction as operation wouldn't be able to leak references to 
outside...


was (Author: xedin):
bq. If I understand you correctly, you mean to deal with the situation where we 
want to flush memtables quick enough that the single cleaner thread may become 
the bottleneck? In which case, I don't think this should ever be a problem: the 
work done by the cleaner itself is very minimal unless it's got some GC to do, 
in which case it is potentially non-trivial, but still unlikely to be a 
bottleneck. There might be a case for handing off the GC candidate selection to 
another thread, except that we expect it to certainly be quicker than any 
flush, and we want to wait for it to complete before deciding if flushing is 
actually worthwhile, so I'm not sure it buys us much. Maybe we could do this 
and continue to check the memory state isn't worsening, and if it is just force 
a flush without waiting for the GC, but it probably is too unlikely to be of 
benefit to be worth the extra complexity. That is, unless you meant some other 
benefit I missed?

I can image the situation when we frequently switch memtables which, in current 
code, starts a new thread, so I wonder how quickly we would be able to see a 
slowdown affect caused by that + potentially OOM on stack space...

bq. The pool is global; a per-cf pool would have the issue of poor sharing of 
resources, though, no? I would probably like to see us move to allocators 
surviving across memtable replacement, so that we could then flatten allocator 
group with allocator, but I'd prefer to save that for a later date.

I'm thinking maybe it would lessen internal complexity if we do (sort-of) arena 
allocation inside instead of having groups and sub pools?

bq. Not really sure what you're suggesting here, could you elaborate?

It's just an idea so people can just ignore it but what I meant there is maybe 
it would be a better first step for us to consider making all of the operations 
that we do more self-contained so we can track exactly where it started and 
ends (by operation I mean all types of reads, writes), which could potentially 
make allocator job easier as we would be able to container  per operation 
memory...

 Partially Off Heap Memtables
 

 Key: CASSANDRA-6689
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 2.1 beta2

[jira] [Updated] (CASSANDRA-6689) Partially Off Heap Memtables