[jira] [Updated] (CASSANDRA-5058) debian packaging should include shuffle
[ https://issues.apache.org/jira/browse/CASSANDRA-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-5058: Component/s: Packaging > debian packaging should include shuffle > --- > > Key: CASSANDRA-5058 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5058 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Brandon Williams >Assignee: Brandon Williams > Fix For: 1.2.0 > > > Our debian packaging doesn't currently include shuffle, but we should add it > so people have a way of upgrading to vnodes. This might also be a good time > to consider a different name for shuffle, though I don't believe it currently > conflicts with anything else. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5058) debian packaging should include shuffle
Brandon Williams created CASSANDRA-5058: --- Summary: debian packaging should include shuffle Key: CASSANDRA-5058 URL: https://issues.apache.org/jira/browse/CASSANDRA-5058 Project: Cassandra Issue Type: Bug Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.2.0 Our debian packaging doesn't currently include shuffle, but we should add it so people have a way of upgrading to vnodes. This might also be a good time to consider a different name for shuffle, though I don't believe it currently conflicts with anything else. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4996) After changing the compaction strategy, compression_strategy always returning back to the "SnappyCompressor" through CQL 2.2.0
[ https://issues.apache.org/jira/browse/CASSANDRA-4996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529407#comment-13529407 ] Aleksey Yeschenko commented on CASSANDRA-4996: -- Basically what [~derek.bromenshenkel] suggested: 1) Remove the default strategy from CFPropDefs 2) Set default strategy in CreateColumnFamilyStatement 3) Leave AlterTableStatement as is (mostly, cql3 version of it needed a slight fix) > After changing the compaction strategy, compression_strategy always > returning back to the "SnappyCompressor" through CQL 2.2.0 > --- > > Key: CASSANDRA-4996 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4996 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.5 >Reporter: Shamim Ahmed >Assignee: Aleksey Yeschenko >Priority: Trivial > Fix For: 1.1.8 > > Attachments: 4996-1.1.txt > > > faced very strange behaviour when changing compression_parameters of > exisiting CF. After changing the compaction strategy, compression_strategy > returning back to the "SnappyCompressor". > Using cassandra version 1.1.5. > [cqlsh 2.2.0 | Cassandra 1.1.5 | CQL spec 2.0.0 | Thrift protocol 19.32.0] > I have one column family with following paramters: > cqlsh > describe columnfamily auditlog_01; > CREATE TABLE auditlog_01 ( > lid text PRIMARY KEY, > dscn text, > asid text, > soapa text > ) WITH > comment='' AND > comparator=text AND > read_repair_chance=0.10 AND > gc_grace_seconds=864000 AND > default_validation=text AND > min_compaction_threshold=4 AND > max_compaction_threshold=32 AND > replicate_on_write='true' AND > compaction_strategy_class='SizeTieredCompactionStrategy' AND > compaction_strategy_options:sstable_size_in_mb='5' AND > compression_parameters:sstable_compression='SnappyCompressor'; > Changing compression strategy to 'DeflateCompressor > cqlsh> ALTER TABLE auditlog_01 WITH compression_parameters:sstabl > e_compression = 'DeflateCompressor' AND > compression_parameters:chunk_length_kb = > 64; > cqlsh> describe columnfamily auditlog_01; > CREATE TABLE auditlog_01 ( > lid text PRIMARY KEY, > dscn text, > asid text, > soapa text > ) WITH > comment='' AND > comparator=text AND > read_repair_chance=0.10 AND > gc_grace_seconds=864000 AND > default_validation=text AND > min_compaction_threshold=4 AND > max_compaction_threshold=32 AND > replicate_on_write='true' AND > compaction_strategy_class='SizeTieredCompactionStrategy' AND > compaction_strategy_options:sstable_size_in_mb='5' AND > compression_parameters:chunk_length_kb='64' AND > compression_parameters:sstable_compression='DeflateCompressor'; > it's sucessfuly changed the compression strategy to 'DeflateCompressor, after > that when i am trying to change the compaction strategy, compression strategy > returing back to "SnappyCompressor". > cqlsh> alter table auditlog_01 with compaction_strategy_class='Le > veledCompactionStrategy' AND compaction_strategy_options:sstable_size_in_mb=5; > cqlsh> describe columnfamily auditlog_01; > CREATE TABLE auditlog_01 ( > lid text PRIMARY KEY, > dscn text, > asid text, > soapa text > ) WITH > comment='' AND > comparator=text AND > read_repair_chance=0.10 AND > gc_grace_seconds=864000 AND > default_validation=text AND > min_compaction_threshold=4 AND > max_compaction_threshold=32 AND > replicate_on_write='true' AND > compaction_strategy_class='SizeTieredCompactionStrategy' AND > compaction_strategy_options:sstable_size_in_mb='5' AND > compression_parameters:sstable_compression='SnappyCompressor'; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4996) After changing the compaction strategy, compression_strategy always returning back to the "SnappyCompressor" through CQL 2.2.0
[ https://issues.apache.org/jira/browse/CASSANDRA-4996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529403#comment-13529403 ] Jonathan Ellis commented on CASSANDRA-4996: --- Can you summarize the fix? > After changing the compaction strategy, compression_strategy always > returning back to the "SnappyCompressor" through CQL 2.2.0 > --- > > Key: CASSANDRA-4996 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4996 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.5 >Reporter: Shamim Ahmed >Assignee: Aleksey Yeschenko >Priority: Trivial > Fix For: 1.1.8 > > Attachments: 4996-1.1.txt > > > faced very strange behaviour when changing compression_parameters of > exisiting CF. After changing the compaction strategy, compression_strategy > returning back to the "SnappyCompressor". > Using cassandra version 1.1.5. > [cqlsh 2.2.0 | Cassandra 1.1.5 | CQL spec 2.0.0 | Thrift protocol 19.32.0] > I have one column family with following paramters: > cqlsh > describe columnfamily auditlog_01; > CREATE TABLE auditlog_01 ( > lid text PRIMARY KEY, > dscn text, > asid text, > soapa text > ) WITH > comment='' AND > comparator=text AND > read_repair_chance=0.10 AND > gc_grace_seconds=864000 AND > default_validation=text AND > min_compaction_threshold=4 AND > max_compaction_threshold=32 AND > replicate_on_write='true' AND > compaction_strategy_class='SizeTieredCompactionStrategy' AND > compaction_strategy_options:sstable_size_in_mb='5' AND > compression_parameters:sstable_compression='SnappyCompressor'; > Changing compression strategy to 'DeflateCompressor > cqlsh> ALTER TABLE auditlog_01 WITH compression_parameters:sstabl > e_compression = 'DeflateCompressor' AND > compression_parameters:chunk_length_kb = > 64; > cqlsh> describe columnfamily auditlog_01; > CREATE TABLE auditlog_01 ( > lid text PRIMARY KEY, > dscn text, > asid text, > soapa text > ) WITH > comment='' AND > comparator=text AND > read_repair_chance=0.10 AND > gc_grace_seconds=864000 AND > default_validation=text AND > min_compaction_threshold=4 AND > max_compaction_threshold=32 AND > replicate_on_write='true' AND > compaction_strategy_class='SizeTieredCompactionStrategy' AND > compaction_strategy_options:sstable_size_in_mb='5' AND > compression_parameters:chunk_length_kb='64' AND > compression_parameters:sstable_compression='DeflateCompressor'; > it's sucessfuly changed the compression strategy to 'DeflateCompressor, after > that when i am trying to change the compaction strategy, compression strategy > returing back to "SnappyCompressor". > cqlsh> alter table auditlog_01 with compaction_strategy_class='Le > veledCompactionStrategy' AND compaction_strategy_options:sstable_size_in_mb=5; > cqlsh> describe columnfamily auditlog_01; > CREATE TABLE auditlog_01 ( > lid text PRIMARY KEY, > dscn text, > asid text, > soapa text > ) WITH > comment='' AND > comparator=text AND > read_repair_chance=0.10 AND > gc_grace_seconds=864000 AND > default_validation=text AND > min_compaction_threshold=4 AND > max_compaction_threshold=32 AND > replicate_on_write='true' AND > compaction_strategy_class='SizeTieredCompactionStrategy' AND > compaction_strategy_options:sstable_size_in_mb='5' AND > compression_parameters:sstable_compression='SnappyCompressor'; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5031) Add ssl support to binary protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-5031: --- Attachment: 0004-CASSANDRA-5031-round4.patch Full patch attached with the recommended changes to SslFactory. > Add ssl support to binary protocol > -- > > Key: CASSANDRA-5031 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5031 > Project: Cassandra > Issue Type: New Feature > Components: API >Reporter: Jonathan Ellis >Assignee: Jason Brown > Fix For: 1.2.1 > > Attachments: > 0001-CASSANDRA-5031-add-ssl-support-to-cql-binary-protoco.patch, > 0002-CASSANDRA-5031_round2.patch, 0003-CASSANDRA-5031-round3.patch, > 0004-CASSANDRA-5031-round4.patch > > > CASSANDRA-4239 added support in Thrift -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4492) HintsColumnFamily compactions hang when using multithreaded compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529288#comment-13529288
]
Michael Kjellman commented on CASSANDRA-4492:
-
no, minimal (if any) deletes.
> HintsColumnFamily compactions hang when using multithreaded compaction
> --
>
> Key: CASSANDRA-4492
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4492
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.0.11
>Reporter: Jason Harvey
>Priority: Minor
> Attachments: jstack.txt
>
>
> Running into an issue on a 6 node ring running 1.0.11 where HintsColumnFamily
> compactions often hang indefinitely when using multithreaded compaction.
> Nothing of note in the logs. In some cases, the compaction hangs before a tmp
> sstable is even created.
> I've wiped out every hints sstable and restarted several times. The issue
> always comes back rather quickly and predictably. The compactions sometimes
> complete if the hint sstables are very small. Disabling multithreaded
> compaction stops this issue from occurring.
> Compactions of all other CFs seem to work just fine.
> This ring was upgraded from 1.0.7. I didn't keep any hints from the upgrade.
> I should note that the ring gets a huge amount of writes, and as a result the
> HintedHandoff rows get be quite wide. I didn't see any large-row compaction
> notices when the compaction was hanging (perhaps the bug was triggered by
> incremental compaction?). After disabling multithreaded compaction, several
> of the rows that were successfully compacted were over 1GB.
> Here is the output I see from compactionstats where a compaction has hung.
> The 'bytes compacted' column never changes.
> {code}
> pending tasks: 1
> compaction typekeyspace column family bytes compacted
> bytes total progress
>Compaction systemHintsColumnFamily 268082
>464784758 0.06%
> {code}
> The hung thread stack is as follows: (full jstack attached, as well)
> {code}
> "CompactionExecutor:37" daemon prio=10 tid=0x063df800 nid=0x49d9
> waiting on condition [0x7eb8c6ffa000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00050f2e0e58> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:329)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:281)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:147)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:126)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:100)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:101)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:88)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:614)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:141)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.call(CompactionManager.java:395)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.ja
[jira] [Commented] (CASSANDRA-4492) HintsColumnFamily compactions hang when using multithreaded compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529284#comment-13529284
]
Thomas Vachon commented on CASSANDRA-4492:
--
No, update heavy though
> HintsColumnFamily compactions hang when using multithreaded compaction
> --
>
> Key: CASSANDRA-4492
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4492
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.0.11
>Reporter: Jason Harvey
>Priority: Minor
> Attachments: jstack.txt
>
>
> Running into an issue on a 6 node ring running 1.0.11 where HintsColumnFamily
> compactions often hang indefinitely when using multithreaded compaction.
> Nothing of note in the logs. In some cases, the compaction hangs before a tmp
> sstable is even created.
> I've wiped out every hints sstable and restarted several times. The issue
> always comes back rather quickly and predictably. The compactions sometimes
> complete if the hint sstables are very small. Disabling multithreaded
> compaction stops this issue from occurring.
> Compactions of all other CFs seem to work just fine.
> This ring was upgraded from 1.0.7. I didn't keep any hints from the upgrade.
> I should note that the ring gets a huge amount of writes, and as a result the
> HintedHandoff rows get be quite wide. I didn't see any large-row compaction
> notices when the compaction was hanging (perhaps the bug was triggered by
> incremental compaction?). After disabling multithreaded compaction, several
> of the rows that were successfully compacted were over 1GB.
> Here is the output I see from compactionstats where a compaction has hung.
> The 'bytes compacted' column never changes.
> {code}
> pending tasks: 1
> compaction typekeyspace column family bytes compacted
> bytes total progress
>Compaction systemHintsColumnFamily 268082
>464784758 0.06%
> {code}
> The hung thread stack is as follows: (full jstack attached, as well)
> {code}
> "CompactionExecutor:37" daemon prio=10 tid=0x063df800 nid=0x49d9
> waiting on condition [0x7eb8c6ffa000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00050f2e0e58> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:329)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:281)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:147)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:126)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:100)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:101)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:88)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:614)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:141)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.call(CompactionManager.java:395)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>
[jira] [Updated] (CASSANDRA-5056) Drop legacy HintsColumnFamily, Migrations, Schema, and LocationInfo CFs after upgrading to 1.2
[ https://issues.apache.org/jira/browse/CASSANDRA-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5056: -- Summary: Drop legacy HintsColumnFamily, Migrations, Schema, and LocationInfo CFs after upgrading to 1.2 (was: Drop legacy HintsColumnFamily, Migrations, and LocationInfo CFs after upgrading to 1.2) > Drop legacy HintsColumnFamily, Migrations, Schema, and LocationInfo CFs after > upgrading to 1.2 > -- > > Key: CASSANDRA-5056 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5056 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Priority: Minor > Fix For: 1.2.1 > > > Once upgraded we don't need to keep these around anymore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4492) HintsColumnFamily compactions hang when using multithreaded compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529271#comment-13529271
]
Jonathan Ellis commented on CASSANDRA-4492:
---
Does your workload also involve truncates [~tvachon] [~mkjellman] [~alienth]?
> HintsColumnFamily compactions hang when using multithreaded compaction
> --
>
> Key: CASSANDRA-4492
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4492
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.0.11
>Reporter: Jason Harvey
>Priority: Minor
> Attachments: jstack.txt
>
>
> Running into an issue on a 6 node ring running 1.0.11 where HintsColumnFamily
> compactions often hang indefinitely when using multithreaded compaction.
> Nothing of note in the logs. In some cases, the compaction hangs before a tmp
> sstable is even created.
> I've wiped out every hints sstable and restarted several times. The issue
> always comes back rather quickly and predictably. The compactions sometimes
> complete if the hint sstables are very small. Disabling multithreaded
> compaction stops this issue from occurring.
> Compactions of all other CFs seem to work just fine.
> This ring was upgraded from 1.0.7. I didn't keep any hints from the upgrade.
> I should note that the ring gets a huge amount of writes, and as a result the
> HintedHandoff rows get be quite wide. I didn't see any large-row compaction
> notices when the compaction was hanging (perhaps the bug was triggered by
> incremental compaction?). After disabling multithreaded compaction, several
> of the rows that were successfully compacted were over 1GB.
> Here is the output I see from compactionstats where a compaction has hung.
> The 'bytes compacted' column never changes.
> {code}
> pending tasks: 1
> compaction typekeyspace column family bytes compacted
> bytes total progress
>Compaction systemHintsColumnFamily 268082
>464784758 0.06%
> {code}
> The hung thread stack is as follows: (full jstack attached, as well)
> {code}
> "CompactionExecutor:37" daemon prio=10 tid=0x063df800 nid=0x49d9
> waiting on condition [0x7eb8c6ffa000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00050f2e0e58> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:329)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:281)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:147)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:126)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:100)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:101)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:88)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:614)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:141)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.call(CompactionManager.java:395)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.
[jira] [Created] (CASSANDRA-5057) Cache reads don't need to be mutable if we're using the off-heap (copying) cache
Jonathan Ellis created CASSANDRA-5057: - Summary: Cache reads don't need to be mutable if we're using the off-heap (copying) cache Key: CASSANDRA-5057 URL: https://issues.apache.org/jira/browse/CASSANDRA-5057 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Trivial Fix For: 1.2.1 Attachments: 5057.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5057) Cache reads don't need to be mutable if we're using the off-heap (copying) cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5057: -- Attachment: 5057.txt patch attached. > Cache reads don't need to be mutable if we're using the off-heap (copying) > cache > > > Key: CASSANDRA-5057 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5057 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Trivial > Fix For: 1.2.1 > > Attachments: 5057.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[2/3] git commit: Optimize name-based queries to use ArrayBackedSortedColumns patch by Pavel Yaskevich; reviewed by slebresne and jbellis for CASSANDRA-5043
Optimize name-based queries to use ArrayBackedSortedColumns
patch by Pavel Yaskevich; reviewed by slebresne and jbellis for CASSANDRA-5043
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db9eb04e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db9eb04e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db9eb04e
Branch: refs/heads/trunk
Commit: db9eb04e2700408303222e1b4bd0096af69399b5
Parents: 31ba289
Author: Jonathan Ellis
Authored: Tue Dec 11 13:44:21 2012 -0600
Committer: Jonathan Ellis
Committed: Tue Dec 11 13:44:21 2012 -0600
--
CHANGES.txt|1 +
.../apache/cassandra/db/CollationController.java | 40 +--
2 files changed, 25 insertions(+), 16 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/db9eb04e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index abb7bad..2a70cec 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
1.2.1
+ * Optimize name-based queries to use ArrayBackedSortedColumns (CASSANDRA-5043)
* Fall back to old manifest if most recent is unparseable (CASSANDRA-5041)
* pool [Compressed]RandomAccessReader objects on the partitioned read path
(CASSANDRA-4942)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/db9eb04e/src/java/org/apache/cassandra/db/CollationController.java
--
diff --git a/src/java/org/apache/cassandra/db/CollationController.java
b/src/java/org/apache/cassandra/db/CollationController.java
index 7160b62..3350de3 100644
--- a/src/java/org/apache/cassandra/db/CollationController.java
+++ b/src/java/org/apache/cassandra/db/CollationController.java
@@ -25,7 +25,6 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.db.columniterator.OnDiskAtomIterator;
-import org.apache.cassandra.db.columniterator.ISSTableColumnIterator;
import org.apache.cassandra.db.columniterator.SimpleAbstractColumnIterator;
import org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy;
import org.apache.cassandra.db.filter.NamesQueryFilter;
@@ -36,14 +35,15 @@ import org.apache.cassandra.io.sstable.SSTableReader;
import org.apache.cassandra.io.util.FileUtils;
import org.apache.cassandra.tracing.Tracing;
import org.apache.cassandra.utils.CloseableIterator;
+import org.apache.cassandra.utils.HeapAllocator;
public class CollationController
{
private static final Logger logger =
LoggerFactory.getLogger(CollationController.class);
private final ColumnFamilyStore cfs;
-private final boolean mutableColumns;
private final QueryFilter filter;
+private final ISortedColumns.Factory factory;
private final int gcBefore;
private int sstablesIterated = 0;
@@ -51,9 +51,13 @@ public class CollationController
public CollationController(ColumnFamilyStore cfs, boolean mutableColumns,
QueryFilter filter, int gcBefore)
{
this.cfs = cfs;
-this.mutableColumns = mutableColumns;
this.filter = filter;
this.gcBefore = gcBefore;
+
+// AtomicSortedColumns doesn't work for super columns (see #3821)
+this.factory = mutableColumns
+ ? cfs.metadata.cfType == ColumnFamilyType.Super ?
ThreadSafeSortedColumns.factory() : AtomicSortedColumns.factory()
+ : ArrayBackedSortedColumns.factory();
}
public ColumnFamily getTopLevelColumns()
@@ -73,15 +77,17 @@ public class CollationController
private ColumnFamily collectTimeOrderedData()
{
logger.trace("collectTimeOrderedData");
-
-// AtomicSortedColumns doesn't work for super columi ns (see #3821)
-ISortedColumns.Factory factory = mutableColumns
- ? cfs.metadata.cfType ==
ColumnFamilyType.Super ? ThreadSafeSortedColumns.factory() :
AtomicSortedColumns.factory()
- : TreeMapBackedSortedColumns.factory();
ColumnFamily container = ColumnFamily.create(cfs.metadata, factory,
filter.filter.isReversed());
List iterators = new
ArrayList();
Tracing.trace("Acquiring sstable references");
ColumnFamilyStore.ViewFragment view = cfs.markReferenced(filter.key);
+
+// We use a temporary CF object per memtable or sstable source so we
can accomodate this.factory being ABSC,
+// which requires addAtom to happen in sorted order. Then we use
addAll to merge into the final collection,
+// which allows a (sorted) set of columns to be merged even if they
are not uniformly sorted after the existing
+// ones.
+ColumnFamily temp = ColumnFamily.create(cf
[3/3] git commit: Optimize name-based queries to use ArrayBackedSortedColumns patch by Pavel Yaskevich; reviewed by slebresne and jbellis for CASSANDRA-5043
Optimize name-based queries to use ArrayBackedSortedColumns
patch by Pavel Yaskevich; reviewed by slebresne and jbellis for CASSANDRA-5043
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db9eb04e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db9eb04e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db9eb04e
Branch: refs/heads/cassandra-1.2
Commit: db9eb04e2700408303222e1b4bd0096af69399b5
Parents: 31ba289
Author: Jonathan Ellis
Authored: Tue Dec 11 13:44:21 2012 -0600
Committer: Jonathan Ellis
Committed: Tue Dec 11 13:44:21 2012 -0600
--
CHANGES.txt|1 +
.../apache/cassandra/db/CollationController.java | 40 +--
2 files changed, 25 insertions(+), 16 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/db9eb04e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index abb7bad..2a70cec 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
1.2.1
+ * Optimize name-based queries to use ArrayBackedSortedColumns (CASSANDRA-5043)
* Fall back to old manifest if most recent is unparseable (CASSANDRA-5041)
* pool [Compressed]RandomAccessReader objects on the partitioned read path
(CASSANDRA-4942)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/db9eb04e/src/java/org/apache/cassandra/db/CollationController.java
--
diff --git a/src/java/org/apache/cassandra/db/CollationController.java
b/src/java/org/apache/cassandra/db/CollationController.java
index 7160b62..3350de3 100644
--- a/src/java/org/apache/cassandra/db/CollationController.java
+++ b/src/java/org/apache/cassandra/db/CollationController.java
@@ -25,7 +25,6 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.db.columniterator.OnDiskAtomIterator;
-import org.apache.cassandra.db.columniterator.ISSTableColumnIterator;
import org.apache.cassandra.db.columniterator.SimpleAbstractColumnIterator;
import org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy;
import org.apache.cassandra.db.filter.NamesQueryFilter;
@@ -36,14 +35,15 @@ import org.apache.cassandra.io.sstable.SSTableReader;
import org.apache.cassandra.io.util.FileUtils;
import org.apache.cassandra.tracing.Tracing;
import org.apache.cassandra.utils.CloseableIterator;
+import org.apache.cassandra.utils.HeapAllocator;
public class CollationController
{
private static final Logger logger =
LoggerFactory.getLogger(CollationController.class);
private final ColumnFamilyStore cfs;
-private final boolean mutableColumns;
private final QueryFilter filter;
+private final ISortedColumns.Factory factory;
private final int gcBefore;
private int sstablesIterated = 0;
@@ -51,9 +51,13 @@ public class CollationController
public CollationController(ColumnFamilyStore cfs, boolean mutableColumns,
QueryFilter filter, int gcBefore)
{
this.cfs = cfs;
-this.mutableColumns = mutableColumns;
this.filter = filter;
this.gcBefore = gcBefore;
+
+// AtomicSortedColumns doesn't work for super columns (see #3821)
+this.factory = mutableColumns
+ ? cfs.metadata.cfType == ColumnFamilyType.Super ?
ThreadSafeSortedColumns.factory() : AtomicSortedColumns.factory()
+ : ArrayBackedSortedColumns.factory();
}
public ColumnFamily getTopLevelColumns()
@@ -73,15 +77,17 @@ public class CollationController
private ColumnFamily collectTimeOrderedData()
{
logger.trace("collectTimeOrderedData");
-
-// AtomicSortedColumns doesn't work for super columi ns (see #3821)
-ISortedColumns.Factory factory = mutableColumns
- ? cfs.metadata.cfType ==
ColumnFamilyType.Super ? ThreadSafeSortedColumns.factory() :
AtomicSortedColumns.factory()
- : TreeMapBackedSortedColumns.factory();
ColumnFamily container = ColumnFamily.create(cfs.metadata, factory,
filter.filter.isReversed());
List iterators = new
ArrayList();
Tracing.trace("Acquiring sstable references");
ColumnFamilyStore.ViewFragment view = cfs.markReferenced(filter.key);
+
+// We use a temporary CF object per memtable or sstable source so we
can accomodate this.factory being ABSC,
+// which requires addAtom to happen in sorted order. Then we use
addAll to merge into the final collection,
+// which allows a (sorted) set of columns to be merged even if they
are not uniformly sorted after the existing
+// ones.
+ColumnFamily temp = ColumnFamily.c
[1/3] git commit: Merge branch 'cassandra-1.2' into trunk
Updated Branches: refs/heads/cassandra-1.2 31ba28909 -> db9eb04e2 refs/heads/trunk 5363189f9 -> bd572d7d9 Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bd572d7d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bd572d7d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bd572d7d Branch: refs/heads/trunk Commit: bd572d7d9b8433565b1afa0aac9106bbc6201b0b Parents: 5363189 db9eb04 Author: Jonathan Ellis Authored: Tue Dec 11 13:44:34 2012 -0600 Committer: Jonathan Ellis Committed: Tue Dec 11 13:44:34 2012 -0600 -- CHANGES.txt|1 + .../apache/cassandra/db/CollationController.java | 39 +-- 2 files changed, 25 insertions(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/bd572d7d/CHANGES.txt -- diff --cc CHANGES.txt index 0cbe968,2a70cec..4585dcc --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,10 -1,5 +1,11 @@@ +1.3 + * make index_interval configurable per columnfamily (CASSANDRA-3961) + * add default_tim_to_live (CASSANDRA-3974) + * add memtable_flush_period_in_ms (CASSANDRA-4237) + + 1.2.1 + * Optimize name-based queries to use ArrayBackedSortedColumns (CASSANDRA-5043) * Fall back to old manifest if most recent is unparseable (CASSANDRA-5041) * pool [Compressed]RandomAccessReader objects on the partitioned read path (CASSANDRA-4942) http://git-wip-us.apache.org/repos/asf/cassandra/blob/bd572d7d/src/java/org/apache/cassandra/db/CollationController.java --
[jira] [Commented] (CASSANDRA-4492) HintsColumnFamily compactions hang when using multithreaded compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529251#comment-13529251
]
Thomas Vachon commented on CASSANDRA-4492:
--
[~jbellis] no I can't. We turned off MT compaction and they have been
compacted away.
> HintsColumnFamily compactions hang when using multithreaded compaction
> --
>
> Key: CASSANDRA-4492
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4492
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.0.11
>Reporter: Jason Harvey
>Priority: Minor
> Attachments: jstack.txt
>
>
> Running into an issue on a 6 node ring running 1.0.11 where HintsColumnFamily
> compactions often hang indefinitely when using multithreaded compaction.
> Nothing of note in the logs. In some cases, the compaction hangs before a tmp
> sstable is even created.
> I've wiped out every hints sstable and restarted several times. The issue
> always comes back rather quickly and predictably. The compactions sometimes
> complete if the hint sstables are very small. Disabling multithreaded
> compaction stops this issue from occurring.
> Compactions of all other CFs seem to work just fine.
> This ring was upgraded from 1.0.7. I didn't keep any hints from the upgrade.
> I should note that the ring gets a huge amount of writes, and as a result the
> HintedHandoff rows get be quite wide. I didn't see any large-row compaction
> notices when the compaction was hanging (perhaps the bug was triggered by
> incremental compaction?). After disabling multithreaded compaction, several
> of the rows that were successfully compacted were over 1GB.
> Here is the output I see from compactionstats where a compaction has hung.
> The 'bytes compacted' column never changes.
> {code}
> pending tasks: 1
> compaction typekeyspace column family bytes compacted
> bytes total progress
>Compaction systemHintsColumnFamily 268082
>464784758 0.06%
> {code}
> The hung thread stack is as follows: (full jstack attached, as well)
> {code}
> "CompactionExecutor:37" daemon prio=10 tid=0x063df800 nid=0x49d9
> waiting on condition [0x7eb8c6ffa000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00050f2e0e58> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:329)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:281)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:147)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:126)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:100)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:101)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:88)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:614)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:141)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.call(CompactionManager.java:395)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>
[jira] [Commented] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529247#comment-13529247 ] Pavel Yaskevich commented on CASSANDRA-5043: bq. ABSC might be faster for 10 columns but there are definitely going to be queries for which log(N) is going to beat N. It would be nice to quantify this somehow in terms of what we know (number of sstables and number of columns being requested for sure; possibly row size and compaction strategy) name slices are not that big e.g. 15-20 as you mentioned, row size is around ~700KB e.g. 50 columns and compaction strategy used is SizeTiredCompactionStrategy, switching to ABSC allowed to cut read latencies almost in half. > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: 5043-v3.txt, > ABSC-with-addAll-for-time-ordered-data.patch, CASSANDRA-5043.patch, > CASSANDRA-5043-v2.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4492) HintsColumnFamily compactions hang when using multithreaded compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529242#comment-13529242
]
T Jake Luciani commented on CASSANDRA-4492:
---
Using 1.2 with LeveledCompaction I have not hit any deadlocks
> HintsColumnFamily compactions hang when using multithreaded compaction
> --
>
> Key: CASSANDRA-4492
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4492
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.0.11
>Reporter: Jason Harvey
>Priority: Minor
> Attachments: jstack.txt
>
>
> Running into an issue on a 6 node ring running 1.0.11 where HintsColumnFamily
> compactions often hang indefinitely when using multithreaded compaction.
> Nothing of note in the logs. In some cases, the compaction hangs before a tmp
> sstable is even created.
> I've wiped out every hints sstable and restarted several times. The issue
> always comes back rather quickly and predictably. The compactions sometimes
> complete if the hint sstables are very small. Disabling multithreaded
> compaction stops this issue from occurring.
> Compactions of all other CFs seem to work just fine.
> This ring was upgraded from 1.0.7. I didn't keep any hints from the upgrade.
> I should note that the ring gets a huge amount of writes, and as a result the
> HintedHandoff rows get be quite wide. I didn't see any large-row compaction
> notices when the compaction was hanging (perhaps the bug was triggered by
> incremental compaction?). After disabling multithreaded compaction, several
> of the rows that were successfully compacted were over 1GB.
> Here is the output I see from compactionstats where a compaction has hung.
> The 'bytes compacted' column never changes.
> {code}
> pending tasks: 1
> compaction typekeyspace column family bytes compacted
> bytes total progress
>Compaction systemHintsColumnFamily 268082
>464784758 0.06%
> {code}
> The hung thread stack is as follows: (full jstack attached, as well)
> {code}
> "CompactionExecutor:37" daemon prio=10 tid=0x063df800 nid=0x49d9
> waiting on condition [0x7eb8c6ffa000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00050f2e0e58> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:329)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:281)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:147)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:126)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:100)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:101)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:88)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:614)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:141)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.call(CompactionManager.java:395)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.
[jira] [Updated] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5043: -- Attachment: 5043-v3.txt Updated comments + formatting for v3. I think there are still a couple important things to address here, though: # ABSC might be faster for 10 columns but there are definitely going to be queries for which log(N) is going to beat N. It would be nice to quantify this somehow in terms of what we know (number of sstables and number of columns being requested for sure; possibly row size and compaction strategy) # this also slows down the mutable container case by introducing the temp container > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: 5043-v3.txt, > ABSC-with-addAll-for-time-ordered-data.patch, CASSANDRA-5043.patch, > CASSANDRA-5043-v2.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529221#comment-13529221 ] Jonathan Ellis commented on CASSANDRA-5043: --- I pulled the caching tweak into 31ba28909fa3cd195a21e0f6702c522fbd712571 (on 1.2 and trunk). > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: ABSC-with-addAll-for-time-ordered-data.patch, > CASSANDRA-5043.patch, CASSANDRA-5043-v2.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[3/3] git commit: add isRowCacheEnabled check to getRawCachedRow; checking cache capacity in getCRI is no longer necessary since it is done by isRowCacheEnabled
add isRowCacheEnabled check to getRawCachedRow; checking cache capacity in
getCRI is no longer necessary since it is done by isRowCacheEnabled
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/31ba2890
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/31ba2890
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/31ba2890
Branch: refs/heads/cassandra-1.2
Commit: 31ba28909fa3cd195a21e0f6702c522fbd712571
Parents: 2d3ee80
Author: Jonathan Ellis
Authored: Tue Dec 11 12:59:42 2012 -0600
Committer: Jonathan Ellis
Committed: Tue Dec 11 12:59:55 2012 -0600
--
.../org/apache/cassandra/db/ColumnFamilyStore.java | 11 +++
1 files changed, 3 insertions(+), 8 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/31ba2890/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 9cb4c66..fa2ae27 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -741,7 +741,7 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
}
// invalidate a normal cache value if it's a sentinel, so the read
will retry (and include the new update)
-IRowCacheEntry cachedRow = getCachedRowInternal(cacheKey);
+IRowCacheEntry cachedRow =
CacheService.instance.rowCache.getInternal(cacheKey);
if (cachedRow != null)
{
if (cachedRow instanceof RowCacheSentinel)
@@ -1653,18 +1653,13 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
*/
public ColumnFamily getRawCachedRow(DecoratedKey key)
{
-if (metadata.cfId == null)
+if (!isRowCacheEnabled() || metadata.cfId == null)
return null; // secondary index
-IRowCacheEntry cached = getCachedRowInternal(new
RowCacheKey(metadata.cfId, key));
+IRowCacheEntry cached = CacheService.instance.rowCache.getInternal(new
RowCacheKey(metadata.cfId, key));
return cached == null || cached instanceof RowCacheSentinel ? null :
(ColumnFamily) cached;
}
-private IRowCacheEntry getCachedRowInternal(RowCacheKey key)
-{
-return CacheService.instance.rowCache.getCapacity() == 0 ? null :
CacheService.instance.rowCache.getInternal(key);
-}
-
/**
* @return true if @param key is contained in the row cache
*/
[1/3] git commit: Merge branch 'cassandra-1.2' into trunk
Updated Branches: refs/heads/cassandra-1.2 2d3ee8000 -> 31ba28909 refs/heads/trunk f7359466a -> 5363189f9 Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5363189f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5363189f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5363189f Branch: refs/heads/trunk Commit: 5363189f9e75c2bc26571448c1b4fb278e297774 Parents: f735946 31ba289 Author: Jonathan Ellis Authored: Tue Dec 11 13:00:08 2012 -0600 Committer: Jonathan Ellis Committed: Tue Dec 11 13:00:08 2012 -0600 -- .../org/apache/cassandra/db/ColumnFamilyStore.java | 11 +++ 1 files changed, 3 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/5363189f/src/java/org/apache/cassandra/db/ColumnFamilyStore.java --
[2/3] git commit: add isRowCacheEnabled check to getRawCachedRow; checking cache capacity in getCRI is no longer necessary since it is done by isRowCacheEnabled
add isRowCacheEnabled check to getRawCachedRow; checking cache capacity in
getCRI is no longer necessary since it is done by isRowCacheEnabled
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/31ba2890
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/31ba2890
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/31ba2890
Branch: refs/heads/trunk
Commit: 31ba28909fa3cd195a21e0f6702c522fbd712571
Parents: 2d3ee80
Author: Jonathan Ellis
Authored: Tue Dec 11 12:59:42 2012 -0600
Committer: Jonathan Ellis
Committed: Tue Dec 11 12:59:55 2012 -0600
--
.../org/apache/cassandra/db/ColumnFamilyStore.java | 11 +++
1 files changed, 3 insertions(+), 8 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/31ba2890/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 9cb4c66..fa2ae27 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -741,7 +741,7 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
}
// invalidate a normal cache value if it's a sentinel, so the read
will retry (and include the new update)
-IRowCacheEntry cachedRow = getCachedRowInternal(cacheKey);
+IRowCacheEntry cachedRow =
CacheService.instance.rowCache.getInternal(cacheKey);
if (cachedRow != null)
{
if (cachedRow instanceof RowCacheSentinel)
@@ -1653,18 +1653,13 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
*/
public ColumnFamily getRawCachedRow(DecoratedKey key)
{
-if (metadata.cfId == null)
+if (!isRowCacheEnabled() || metadata.cfId == null)
return null; // secondary index
-IRowCacheEntry cached = getCachedRowInternal(new
RowCacheKey(metadata.cfId, key));
+IRowCacheEntry cached = CacheService.instance.rowCache.getInternal(new
RowCacheKey(metadata.cfId, key));
return cached == null || cached instanceof RowCacheSentinel ? null :
(ColumnFamily) cached;
}
-private IRowCacheEntry getCachedRowInternal(RowCacheKey key)
-{
-return CacheService.instance.rowCache.getCapacity() == 0 ? null :
CacheService.instance.rowCache.getInternal(key);
-}
-
/**
* @return true if @param key is contained in the row cache
*/
[jira] [Resolved] (CASSANDRA-4997) Remove multithreaded_compaction from cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-4997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-4997. --- Resolution: Fixed Fix Version/s: 1.2.0 rc1 1.1.8 Assignee: Jonathan Ellis added a warning referencing CASSANDRA-4492 instead of removing it entirely. > Remove multithreaded_compaction from cassandra.yaml > --- > > Key: CASSANDRA-4997 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4997 > Project: Cassandra > Issue Type: Task > Components: Core >Affects Versions: 1.0.12, 1.1.6 >Reporter: Jeremy Hanna >Assignee: Jonathan Ellis >Priority: Trivial > Fix For: 1.1.8, 1.2.0 rc1 > > > There have been several issues with multithreaded_compaction and it's > generally discouraged for use, even on SSD based systems. It seems like it > would be a good idea to remove it completely from the cassandra.yaml but > leave it as a setting for now. That way people can manually add it, but > unsuspecting people who don't know about the sharp edges won't enable it on > their SSD based systems. > An alternative to removing it from the cassandra.yaml is to add more warnings > to the description. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[10/14] git commit: add warning to multithreaded_compaction
add warning to multithreaded_compaction Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ca02176f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ca02176f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ca02176f Branch: refs/heads/cassandra-1.2 Commit: ca02176fc0cde6eee6b0f278ece58ce8ce202d18 Parents: 36b183f Author: Jonathan Ellis Authored: Tue Dec 11 12:37:47 2012 -0600 Committer: Jonathan Ellis Committed: Tue Dec 11 12:37:47 2012 -0600 -- conf/cassandra.yaml | 15 +-- 1 files changed, 9 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ca02176f/conf/cassandra.yaml -- diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index c4732db..543ad65 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -381,12 +381,15 @@ in_memory_compaction_limit_in_mb: 64 # Uncomment to make compaction mono-threaded, the pre-0.8 default. #concurrent_compactors: 1 -# Multi-threaded compaction. When enabled, each compaction will use -# up to one thread per core, plus one thread per sstable being merged. -# This is usually only useful for SSD-based hardware: otherwise, -# your concern is usually to get compaction to do LESS i/o (see: -# compaction_throughput_mb_per_sec), not more. -multithreaded_compaction: false +# multithreaded_compaction: false. When enabled, each compaction will +# use up to one thread per core, plus one thread per sstable being +# merged. This is usually only useful for SSD-based hardware: +# otherwise, your concern is usually to get compaction to do LESS i/o +# (see: compaction_throughput_mb_per_sec), not more. +# +# WARNING: this setting has caused compaction deadlocks for multiple +# users (see CASSANDRA-4492). It is recommended to leave this off +# unless you are prepared to help troubleshoot. # Throttles compaction to the given total throughput across the entire # system. The faster you insert data, the faster you need to compact in
[11/14] git commit: add warning to multithreaded_compaction
add warning to multithreaded_compaction Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ca02176f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ca02176f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ca02176f Branch: refs/heads/trunk Commit: ca02176fc0cde6eee6b0f278ece58ce8ce202d18 Parents: 36b183f Author: Jonathan Ellis Authored: Tue Dec 11 12:37:47 2012 -0600 Committer: Jonathan Ellis Committed: Tue Dec 11 12:37:47 2012 -0600 -- conf/cassandra.yaml | 15 +-- 1 files changed, 9 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ca02176f/conf/cassandra.yaml -- diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index c4732db..543ad65 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -381,12 +381,15 @@ in_memory_compaction_limit_in_mb: 64 # Uncomment to make compaction mono-threaded, the pre-0.8 default. #concurrent_compactors: 1 -# Multi-threaded compaction. When enabled, each compaction will use -# up to one thread per core, plus one thread per sstable being merged. -# This is usually only useful for SSD-based hardware: otherwise, -# your concern is usually to get compaction to do LESS i/o (see: -# compaction_throughput_mb_per_sec), not more. -multithreaded_compaction: false +# multithreaded_compaction: false. When enabled, each compaction will +# use up to one thread per core, plus one thread per sstable being +# merged. This is usually only useful for SSD-based hardware: +# otherwise, your concern is usually to get compaction to do LESS i/o +# (see: compaction_throughput_mb_per_sec), not more. +# +# WARNING: this setting has caused compaction deadlocks for multiple +# users (see CASSANDRA-4492). It is recommended to leave this off +# unless you are prepared to help troubleshoot. # Throttles compaction to the given total throughput across the entire # system. The faster you insert data, the faster you need to compact in
[13/14] git commit: Update version for 1.2.0-rc1 release
Update version for 1.2.0-rc1 release Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d791e0b3 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d791e0b3 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d791e0b3 Branch: refs/heads/trunk Commit: d791e0b3fa5a615f2df200ecd40f82dc9a5874d6 Parents: a399a56 Author: Sylvain Lebresne Authored: Tue Dec 11 09:10:45 2012 +0100 Committer: Sylvain Lebresne Committed: Tue Dec 11 09:10:45 2012 +0100 -- build.xml|2 +- debian/changelog |6 ++ 2 files changed, 7 insertions(+), 1 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d791e0b3/build.xml -- diff --git a/build.xml b/build.xml index 1f75261..e9b8e78 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> http://git-wip-us.apache.org/repos/asf/cassandra/blob/d791e0b3/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index 3bb10aa..cfa1781 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (1.2.0~rc1) unstable; urgency=low + + * New RC release + + -- Sylvain Lebresne Tue, 11 Dec 2012 09:09:08 +0100 + cassandra (1.2.0~beta3) unstable; urgency=low * New beta release
[14/14] git commit: Update version for 1.2.0-rc1 release
Update version for 1.2.0-rc1 release Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d791e0b3 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d791e0b3 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d791e0b3 Branch: refs/heads/cassandra-1.2 Commit: d791e0b3fa5a615f2df200ecd40f82dc9a5874d6 Parents: a399a56 Author: Sylvain Lebresne Authored: Tue Dec 11 09:10:45 2012 +0100 Committer: Sylvain Lebresne Committed: Tue Dec 11 09:10:45 2012 +0100 -- build.xml|2 +- debian/changelog |6 ++ 2 files changed, 7 insertions(+), 1 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d791e0b3/build.xml -- diff --git a/build.xml b/build.xml index 1f75261..e9b8e78 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> http://git-wip-us.apache.org/repos/asf/cassandra/blob/d791e0b3/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index 3bb10aa..cfa1781 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (1.2.0~rc1) unstable; urgency=low + + * New RC release + + -- Sylvain Lebresne Tue, 11 Dec 2012 09:09:08 +0100 + cassandra (1.2.0~beta3) unstable; urgency=low * New beta release
[9/14] git commit: add warning to multithreaded_compaction
add warning to multithreaded_compaction Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ca02176f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ca02176f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ca02176f Branch: refs/heads/cassandra-1.2.0 Commit: ca02176fc0cde6eee6b0f278ece58ce8ce202d18 Parents: 36b183f Author: Jonathan Ellis Authored: Tue Dec 11 12:37:47 2012 -0600 Committer: Jonathan Ellis Committed: Tue Dec 11 12:37:47 2012 -0600 -- conf/cassandra.yaml | 15 +-- 1 files changed, 9 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ca02176f/conf/cassandra.yaml -- diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index c4732db..543ad65 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -381,12 +381,15 @@ in_memory_compaction_limit_in_mb: 64 # Uncomment to make compaction mono-threaded, the pre-0.8 default. #concurrent_compactors: 1 -# Multi-threaded compaction. When enabled, each compaction will use -# up to one thread per core, plus one thread per sstable being merged. -# This is usually only useful for SSD-based hardware: otherwise, -# your concern is usually to get compaction to do LESS i/o (see: -# compaction_throughput_mb_per_sec), not more. -multithreaded_compaction: false +# multithreaded_compaction: false. When enabled, each compaction will +# use up to one thread per core, plus one thread per sstable being +# merged. This is usually only useful for SSD-based hardware: +# otherwise, your concern is usually to get compaction to do LESS i/o +# (see: compaction_throughput_mb_per_sec), not more. +# +# WARNING: this setting has caused compaction deadlocks for multiple +# users (see CASSANDRA-4492). It is recommended to leave this off +# unless you are prepared to help troubleshoot. # Throttles compaction to the given total throughput across the entire # system. The faster you insert data, the faster you need to compact in
[12/14] git commit: add warning to multithreaded_compaction
add warning to multithreaded_compaction Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ca02176f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ca02176f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ca02176f Branch: refs/heads/cassandra-1.1 Commit: ca02176fc0cde6eee6b0f278ece58ce8ce202d18 Parents: 36b183f Author: Jonathan Ellis Authored: Tue Dec 11 12:37:47 2012 -0600 Committer: Jonathan Ellis Committed: Tue Dec 11 12:37:47 2012 -0600 -- conf/cassandra.yaml | 15 +-- 1 files changed, 9 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ca02176f/conf/cassandra.yaml -- diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index c4732db..543ad65 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -381,12 +381,15 @@ in_memory_compaction_limit_in_mb: 64 # Uncomment to make compaction mono-threaded, the pre-0.8 default. #concurrent_compactors: 1 -# Multi-threaded compaction. When enabled, each compaction will use -# up to one thread per core, plus one thread per sstable being merged. -# This is usually only useful for SSD-based hardware: otherwise, -# your concern is usually to get compaction to do LESS i/o (see: -# compaction_throughput_mb_per_sec), not more. -multithreaded_compaction: false +# multithreaded_compaction: false. When enabled, each compaction will +# use up to one thread per core, plus one thread per sstable being +# merged. This is usually only useful for SSD-based hardware: +# otherwise, your concern is usually to get compaction to do LESS i/o +# (see: compaction_throughput_mb_per_sec), not more. +# +# WARNING: this setting has caused compaction deadlocks for multiple +# users (see CASSANDRA-4492). It is recommended to leave this off +# unless you are prepared to help troubleshoot. # Throttles compaction to the given total throughput across the entire # system. The faster you insert data, the faster you need to compact in
[8/14] git commit: Merge branch 'cassandra-1.1' into cassandra-1.2.0
Merge branch 'cassandra-1.1' into cassandra-1.2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fc5a0cc2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fc5a0cc2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fc5a0cc2 Branch: refs/heads/cassandra-1.2.0 Commit: fc5a0cc293adad16050d3dc232495439a42d6bfe Parents: d791e0b ca02176 Author: Jonathan Ellis Authored: Tue Dec 11 12:37:55 2012 -0600 Committer: Jonathan Ellis Committed: Tue Dec 11 12:37:55 2012 -0600 -- conf/cassandra.yaml | 15 +-- 1 files changed, 9 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fc5a0cc2/conf/cassandra.yaml --
[3/14] git commit: Merge branch 'cassandra-1.2.0' into cassandra-1.2
Merge branch 'cassandra-1.2.0' into cassandra-1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2d3ee800 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2d3ee800 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2d3ee800 Branch: refs/heads/cassandra-1.2 Commit: 2d3ee8000e14e0060ba7ee23ade5a207c6bb04e2 Parents: dfe3ef9 fc5a0cc Author: Jonathan Ellis Authored: Tue Dec 11 12:38:05 2012 -0600 Committer: Jonathan Ellis Committed: Tue Dec 11 12:38:05 2012 -0600 -- build.xml |2 +- conf/cassandra.yaml | 15 +-- debian/changelog|6 ++ 3 files changed, 16 insertions(+), 7 deletions(-) --
[5/14] git commit: r/m unused field
r/m unused field
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dfe3ef9a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dfe3ef9a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dfe3ef9a
Branch: refs/heads/cassandra-1.2
Commit: dfe3ef9a3b7a9c81585e1a6c22f74d4026155b28
Parents: df580d4
Author: Jonathan Ellis
Authored: Tue Dec 11 11:21:59 2012 -0600
Committer: Jonathan Ellis
Committed: Tue Dec 11 12:38:02 2012 -0600
--
src/java/org/apache/cassandra/db/SystemTable.java |1 -
1 files changed, 0 insertions(+), 1 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/dfe3ef9a/src/java/org/apache/cassandra/db/SystemTable.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemTable.java
b/src/java/org/apache/cassandra/db/SystemTable.java
index bfd7ac9..8f60904 100644
--- a/src/java/org/apache/cassandra/db/SystemTable.java
+++ b/src/java/org/apache/cassandra/db/SystemTable.java
@@ -84,7 +84,6 @@ public class SystemTable
public static final String OLD_HINTS_CF = "HintsColumnFamily";
private static final String LOCAL_KEY = "local";
-private static final ByteBuffer CURRENT_LOCAL_NODE_ID_KEY =
ByteBufferUtil.bytes("CurrentLocal");
private static final ByteBuffer ALL_LOCAL_NODE_ID_KEY =
ByteBufferUtil.bytes("Local");
public enum BootstrapState
[1/14] git commit: Merge branch 'cassandra-1.2' into trunk
Updated Branches: refs/heads/cassandra-1.1 36b183f80 -> ca02176fc refs/heads/cassandra-1.2 df580d426 -> 2d3ee8000 refs/heads/cassandra-1.2.0 d791e0b3f -> fc5a0cc29 refs/heads/trunk 647ef5aeb -> f7359466a Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f7359466 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f7359466 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f7359466 Branch: refs/heads/trunk Commit: f7359466a10281fb193287601dc9fc48399c4e81 Parents: 647ef5a 2d3ee80 Author: Jonathan Ellis Authored: Tue Dec 11 12:38:19 2012 -0600 Committer: Jonathan Ellis Committed: Tue Dec 11 12:38:19 2012 -0600 -- build.xml |2 +- conf/cassandra.yaml | 15 +-- debian/changelog |6 ++ src/java/org/apache/cassandra/db/SystemTable.java |1 - 4 files changed, 16 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7359466/conf/cassandra.yaml --
[7/14] git commit: Merge branch 'cassandra-1.1' into cassandra-1.2.0
Merge branch 'cassandra-1.1' into cassandra-1.2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fc5a0cc2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fc5a0cc2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fc5a0cc2 Branch: refs/heads/trunk Commit: fc5a0cc293adad16050d3dc232495439a42d6bfe Parents: d791e0b ca02176 Author: Jonathan Ellis Authored: Tue Dec 11 12:37:55 2012 -0600 Committer: Jonathan Ellis Committed: Tue Dec 11 12:37:55 2012 -0600 -- conf/cassandra.yaml | 15 +-- 1 files changed, 9 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fc5a0cc2/conf/cassandra.yaml --
[2/14] git commit: Merge branch 'cassandra-1.2.0' into cassandra-1.2
Merge branch 'cassandra-1.2.0' into cassandra-1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2d3ee800 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2d3ee800 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2d3ee800 Branch: refs/heads/trunk Commit: 2d3ee8000e14e0060ba7ee23ade5a207c6bb04e2 Parents: dfe3ef9 fc5a0cc Author: Jonathan Ellis Authored: Tue Dec 11 12:38:05 2012 -0600 Committer: Jonathan Ellis Committed: Tue Dec 11 12:38:05 2012 -0600 -- build.xml |2 +- conf/cassandra.yaml | 15 +-- debian/changelog|6 ++ 3 files changed, 16 insertions(+), 7 deletions(-) --
[6/14] git commit: Merge branch 'cassandra-1.1' into cassandra-1.2.0
Merge branch 'cassandra-1.1' into cassandra-1.2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fc5a0cc2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fc5a0cc2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fc5a0cc2 Branch: refs/heads/cassandra-1.2 Commit: fc5a0cc293adad16050d3dc232495439a42d6bfe Parents: d791e0b ca02176 Author: Jonathan Ellis Authored: Tue Dec 11 12:37:55 2012 -0600 Committer: Jonathan Ellis Committed: Tue Dec 11 12:37:55 2012 -0600 -- conf/cassandra.yaml | 15 +-- 1 files changed, 9 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fc5a0cc2/conf/cassandra.yaml --
[4/14] git commit: r/m unused field
r/m unused field
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dfe3ef9a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dfe3ef9a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dfe3ef9a
Branch: refs/heads/trunk
Commit: dfe3ef9a3b7a9c81585e1a6c22f74d4026155b28
Parents: df580d4
Author: Jonathan Ellis
Authored: Tue Dec 11 11:21:59 2012 -0600
Committer: Jonathan Ellis
Committed: Tue Dec 11 12:38:02 2012 -0600
--
src/java/org/apache/cassandra/db/SystemTable.java |1 -
1 files changed, 0 insertions(+), 1 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/dfe3ef9a/src/java/org/apache/cassandra/db/SystemTable.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemTable.java
b/src/java/org/apache/cassandra/db/SystemTable.java
index bfd7ac9..8f60904 100644
--- a/src/java/org/apache/cassandra/db/SystemTable.java
+++ b/src/java/org/apache/cassandra/db/SystemTable.java
@@ -84,7 +84,6 @@ public class SystemTable
public static final String OLD_HINTS_CF = "HintsColumnFamily";
private static final String LOCAL_KEY = "local";
-private static final ByteBuffer CURRENT_LOCAL_NODE_ID_KEY =
ByteBufferUtil.bytes("CurrentLocal");
private static final ByteBuffer ALL_LOCAL_NODE_ID_KEY =
ByteBufferUtil.bytes("Local");
public enum BootstrapState
[jira] [Updated] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-5043: --- Attachment: CASSANDRA-5043-v2.patch Sorry, it didn't apply because you had to first apply CASSANDRA-5043.patch. Attaching v2 with combines two patches for 1.2.0 branch and has reduceNameFilter fixed to use container. I don't think we should bother with one column optimization especially as there are iterators involved, that would just add complexity to the code. > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: ABSC-with-addAll-for-time-ordered-data.patch, > CASSANDRA-5043.patch, CASSANDRA-5043-v2.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5056) Drop legacy HintsColumnFamily, Migrations, and LocationInfo CFs after upgrading to 1.2
Jonathan Ellis created CASSANDRA-5056: - Summary: Drop legacy HintsColumnFamily, Migrations, and LocationInfo CFs after upgrading to 1.2 Key: CASSANDRA-5056 URL: https://issues.apache.org/jira/browse/CASSANDRA-5056 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.0 beta 1 Reporter: Jonathan Ellis Priority: Minor Fix For: 1.2.1 Once upgraded we don't need to keep these around anymore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5054) new partitioner for rowkey pairs (P1, P2) targeting hierarchical data
[ https://issues.apache.org/jira/browse/CASSANDRA-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529135#comment-13529135 ] Dominique De Vito commented on CASSANDRA-5054: -- In that case, AFAIK, your proposal is about to replace rowkey like (P1, P2) by rowkey P1 while repeating P2 as 1st part of composite column names. We didn't followed that idea for 2 reasons: 1) P2 could be quite long (it's unfortunately a string, not a 'long' integer), we try to avoid to repeat it 2) we wanted some fixed column names (so, without varying P2 as 1st part of a composite column name) for secondary index goals So, this new partitioner appeared (IMHO) to us as a better design than wide row. > new partitioner for rowkey pairs (P1, P2) targeting hierarchical data > - > > Key: CASSANDRA-5054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5054 > Project: Cassandra > Issue Type: Improvement > Components: Contrib, Core >Reporter: Dominique De Vito >Priority: Trivial > Attachments: RandomPartitioner4Pair.java > > > This new partitioner is submitted here both for validation and for proposal > to other Cassandra users (may be this partitioner has its place within > Cassandra core, or into some 'contrib' directory). > This new partitioner is a variant of RandomPartitioner with special token > computation for composite rowkeys of size 2. > The use case of this partitioner is about rowkeys of hierarchical data, that > is, rowkeys like (directory id, file id). > This partitioner computes tokens so that rows with same "directory id" have a > great chance to be on the same node: the goal is that, when Cassandra is > asked about multiple file ids for the same directory id, a limited number of > nodes should be asked. > So, in case of a composite rowkey of size 2 like (P1, P2), the partitioner > computes the token as follows: merge(getHighBits(md5(P1)), > getLowBits(md5(P2))). > In case of a rowkey that is NOT a pair, the partitioner returns the same > value than RandomPartitioner. > This partitioner is expected to be used with Cassandra 1.1 or above. > Cassandra stores in sstables the pair (token, rowkey). Since v1.1, during a > compaction phase, Cassandra does not sort anymore on token only, but > sort on the pair (token, rowkey). So, if a custom partitioner produces token > collisions (that is, multiple rowkeys with the same token), it won't be a pb > with v1.1 because, in order to retrieve a row, Cassandra is going to compare > the full pair (token, rowkey). > As the percent of row collisions for this partitioner is unknown, it's > expected to be used with Cassandra 1.1 or above. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5053) not possible to change crc_check_chance
[ https://issues.apache.org/jira/browse/CASSANDRA-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529133#comment-13529133 ] Marcus Eriksson commented on CASSANDRA-5053: for this i actually prefer a config, then i can tweak it on one machine at a time and since it has been broken since 1.1.1 i doubt anyone is actually using it > not possible to change crc_check_chance > --- > > Key: CASSANDRA-5053 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5053 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.1, 1.1.7 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.1.7 > > Attachments: > 0001-fix-CASSANDRA-5053-not-possible-to-change-crc_check_.patch > > > It is not possible to change crc_check_chance using a schema modification > after CASSANDRA-4266 > This patch fixes that and moves the setting out into a configuration > parameter instead, you dont want to upgrade/scrub/.. all your sstables to > change the crc_check_chance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4492) HintsColumnFamily compactions hang when using multithreaded compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529123#comment-13529123
]
Jonathan Ellis commented on CASSANDRA-4492:
---
bq. If you call truncate while a compaction is currently going on it hangs
both the truncate and the parallel compaction iterator.
Truncate took some big changes for 1.2 (CASSANDRA-4096) so I doubt this is
still the case. (If it is, I'm not sure it's related to compaction alone
causing the hang.)
> HintsColumnFamily compactions hang when using multithreaded compaction
> --
>
> Key: CASSANDRA-4492
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4492
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.0.11
>Reporter: Jason Harvey
>Priority: Minor
> Attachments: jstack.txt
>
>
> Running into an issue on a 6 node ring running 1.0.11 where HintsColumnFamily
> compactions often hang indefinitely when using multithreaded compaction.
> Nothing of note in the logs. In some cases, the compaction hangs before a tmp
> sstable is even created.
> I've wiped out every hints sstable and restarted several times. The issue
> always comes back rather quickly and predictably. The compactions sometimes
> complete if the hint sstables are very small. Disabling multithreaded
> compaction stops this issue from occurring.
> Compactions of all other CFs seem to work just fine.
> This ring was upgraded from 1.0.7. I didn't keep any hints from the upgrade.
> I should note that the ring gets a huge amount of writes, and as a result the
> HintedHandoff rows get be quite wide. I didn't see any large-row compaction
> notices when the compaction was hanging (perhaps the bug was triggered by
> incremental compaction?). After disabling multithreaded compaction, several
> of the rows that were successfully compacted were over 1GB.
> Here is the output I see from compactionstats where a compaction has hung.
> The 'bytes compacted' column never changes.
> {code}
> pending tasks: 1
> compaction typekeyspace column family bytes compacted
> bytes total progress
>Compaction systemHintsColumnFamily 268082
>464784758 0.06%
> {code}
> The hung thread stack is as follows: (full jstack attached, as well)
> {code}
> "CompactionExecutor:37" daemon prio=10 tid=0x063df800 nid=0x49d9
> waiting on condition [0x7eb8c6ffa000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00050f2e0e58> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:329)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:281)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:147)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:126)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:100)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:101)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:88)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:614)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.CompactionTask.execute(
[jira] [Commented] (CASSANDRA-4492) HintsColumnFamily compactions hang when using multithreaded compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529111#comment-13529111
]
Jonathan Ellis commented on CASSANDRA-4492:
---
If you can give us a set of sstables that reliably cause the hang (snapshot
before compaction option should be useful here) then we can troubleshoot.
Nothing is obviously wrong from just eyeballing things.
> HintsColumnFamily compactions hang when using multithreaded compaction
> --
>
> Key: CASSANDRA-4492
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4492
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.0.11
>Reporter: Jason Harvey
>Priority: Minor
> Attachments: jstack.txt
>
>
> Running into an issue on a 6 node ring running 1.0.11 where HintsColumnFamily
> compactions often hang indefinitely when using multithreaded compaction.
> Nothing of note in the logs. In some cases, the compaction hangs before a tmp
> sstable is even created.
> I've wiped out every hints sstable and restarted several times. The issue
> always comes back rather quickly and predictably. The compactions sometimes
> complete if the hint sstables are very small. Disabling multithreaded
> compaction stops this issue from occurring.
> Compactions of all other CFs seem to work just fine.
> This ring was upgraded from 1.0.7. I didn't keep any hints from the upgrade.
> I should note that the ring gets a huge amount of writes, and as a result the
> HintedHandoff rows get be quite wide. I didn't see any large-row compaction
> notices when the compaction was hanging (perhaps the bug was triggered by
> incremental compaction?). After disabling multithreaded compaction, several
> of the rows that were successfully compacted were over 1GB.
> Here is the output I see from compactionstats where a compaction has hung.
> The 'bytes compacted' column never changes.
> {code}
> pending tasks: 1
> compaction typekeyspace column family bytes compacted
> bytes total progress
>Compaction systemHintsColumnFamily 268082
>464784758 0.06%
> {code}
> The hung thread stack is as follows: (full jstack attached, as well)
> {code}
> "CompactionExecutor:37" daemon prio=10 tid=0x063df800 nid=0x49d9
> waiting on condition [0x7eb8c6ffa000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00050f2e0e58> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:329)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer.computeNext(ParallelCompactionIterable.java:281)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:147)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:126)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:100)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:101)
> at
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Unwrapper.computeNext(ParallelCompactionIterable.java:88)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:614)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at
> org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:141)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.
[jira] [Commented] (CASSANDRA-5053) not possible to change crc_check_chance
[ https://issues.apache.org/jira/browse/CASSANDRA-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529100#comment-13529100 ] Jonathan Ellis commented on CASSANDRA-5053: --- Moving it from per-CF to global seems kind of backwards, don't you think? The overwhelming tendency has been for people to push for more fine-grained control over things. > not possible to change crc_check_chance > --- > > Key: CASSANDRA-5053 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5053 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.1, 1.1.7 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.1.7 > > Attachments: > 0001-fix-CASSANDRA-5053-not-possible-to-change-crc_check_.patch > > > It is not possible to change crc_check_chance using a schema modification > after CASSANDRA-4266 > This patch fixes that and moves the setting out into a configuration > parameter instead, you dont want to upgrade/scrub/.. all your sstables to > change the crc_check_chance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5054) new partitioner for rowkey pairs (P1, P2) targeting hierarchical data
[ https://issues.apache.org/jira/browse/CASSANDRA-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529097#comment-13529097 ] Jonathan Ellis commented on CASSANDRA-5054: --- My first reaction is that we probably don't want to encourage people doing this vs a single wide row. What problem with the latter are you trying to address? > new partitioner for rowkey pairs (P1, P2) targeting hierarchical data > - > > Key: CASSANDRA-5054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5054 > Project: Cassandra > Issue Type: Improvement > Components: Contrib, Core >Reporter: Dominique De Vito >Priority: Trivial > Attachments: RandomPartitioner4Pair.java > > > This new partitioner is submitted here both for validation and for proposal > to other Cassandra users (may be this partitioner has its place within > Cassandra core, or into some 'contrib' directory). > This new partitioner is a variant of RandomPartitioner with special token > computation for composite rowkeys of size 2. > The use case of this partitioner is about rowkeys of hierarchical data, that > is, rowkeys like (directory id, file id). > This partitioner computes tokens so that rows with same "directory id" have a > great chance to be on the same node: the goal is that, when Cassandra is > asked about multiple file ids for the same directory id, a limited number of > nodes should be asked. > So, in case of a composite rowkey of size 2 like (P1, P2), the partitioner > computes the token as follows: merge(getHighBits(md5(P1)), > getLowBits(md5(P2))). > In case of a rowkey that is NOT a pair, the partitioner returns the same > value than RandomPartitioner. > This partitioner is expected to be used with Cassandra 1.1 or above. > Cassandra stores in sstables the pair (token, rowkey). Since v1.1, during a > compaction phase, Cassandra does not sort anymore on token only, but > sort on the pair (token, rowkey). So, if a custom partitioner produces token > collisions (that is, multiple rowkeys with the same token), it won't be a pb > with v1.1 because, in order to retrieve a row, Cassandra is going to compare > the full pair (token, rowkey). > As the percent of row collisions for this partitioner is unknown, it's > expected to be used with Cassandra 1.1 or above. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-5055) Support JDO
[ https://issues.apache.org/jira/browse/CASSANDRA-5055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-5055. --- Resolution: Won't Fix Cassandra developers are happy to provide advice but are not interested in creating or maintaining this in-tree. > Support JDO > --- > > Key: CASSANDRA-5055 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5055 > Project: Cassandra > Issue Type: New Feature >Reporter: Matthew T. Adams > Labels: datanucleus, driver, jdo, plugin > > JDO is a datastore-agnostic Java standard for transparent persistence that > precedes and is largely a functional superset of JPA. JDO's API & TCK is > maintained by the Apache DB project and can be found at > http://db.apache.org/jdo. The reference implementation is DataNucleus, which > can be found at http://www.datanucleus.org. > This request is to have Cassandra provide an officially supported native JDO > implementation or Cassandra plugin for DataNucleus. > There is some past work available on github.com involving JDO, DataNucleus > and Cassandra, which might be a good starting point for this work: > https://github.com/search?q=datanucleus+cassandra -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5055) Support JDO
Matthew T. Adams created CASSANDRA-5055: --- Summary: Support JDO Key: CASSANDRA-5055 URL: https://issues.apache.org/jira/browse/CASSANDRA-5055 Project: Cassandra Issue Type: New Feature Reporter: Matthew T. Adams JDO is a datastore-agnostic Java standard for transparent persistence that precedes and is largely a functional superset of JPA. JDO's API & TCK is maintained by the Apache DB project and can be found at http://db.apache.org/jdo. The reference implementation is DataNucleus, which can be found at http://www.datanucleus.org. This request is to have Cassandra provide an officially supported native JDO implementation or Cassandra plugin for DataNucleus. There is some past work available on github.com involving JDO, DataNucleus and Cassandra, which might be a good starting point for this work: https://github.com/search?q=datanucleus+cassandra -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5038) LZ4Compressor
[ https://issues.apache.org/jira/browse/CASSANDRA-5038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-5038: -- Attachment: LZ4Compressor.java lz4-java.jar New version that encodes the length of each chunk in first 4 bytes (didn't use vints) Uses known decompress. Also uses the fastestInstance call now > LZ4Compressor > - > > Key: CASSANDRA-5038 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5038 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: T Jake Luciani >Priority: Minor > Fix For: 1.2.1 > > Attachments: LZ4Compressor.java, lz4-java.jar > > > LZ4 is a new compression algo that's ~2x faster than Snappy. > [~jpountz] has written a nice java port which includes a misc.Unsafe version > that performs >= than our java snappy version. > Details at http://blog.jpountz.net/post/28092106032/wow-lz4-is-fast > The nice thing is this should work with java7 and be more portable. > We can also fallback the pure java impl -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5038) LZ4Compressor
[ https://issues.apache.org/jira/browse/CASSANDRA-5038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-5038: -- Attachment: (was: lz4-java.jar) > LZ4Compressor > - > > Key: CASSANDRA-5038 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5038 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: T Jake Luciani >Priority: Minor > Fix For: 1.2.1 > > > LZ4 is a new compression algo that's ~2x faster than Snappy. > [~jpountz] has written a nice java port which includes a misc.Unsafe version > that performs >= than our java snappy version. > Details at http://blog.jpountz.net/post/28092106032/wow-lz4-is-fast > The nice thing is this should work with java7 and be more portable. > We can also fallback the pure java impl -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5038) LZ4Compressor
[ https://issues.apache.org/jira/browse/CASSANDRA-5038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-5038: -- Attachment: (was: LZ4Compressor.java) > LZ4Compressor > - > > Key: CASSANDRA-5038 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5038 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: T Jake Luciani >Priority: Minor > Fix For: 1.2.1 > > > LZ4 is a new compression algo that's ~2x faster than Snappy. > [~jpountz] has written a nice java port which includes a misc.Unsafe version > that performs >= than our java snappy version. > Details at http://blog.jpountz.net/post/28092106032/wow-lz4-is-fast > The nice thing is this should work with java7 and be more portable. > We can also fallback the pure java impl -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5020) Time to switch back to byte[] internally?
[ https://issues.apache.org/jira/browse/CASSANDRA-5020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529052#comment-13529052 ] Jonathan Ellis edited comment on CASSANDRA-5020 at 12/11/12 3:29 PM: - I don't think switching to byte[] is crazy either. We'd save over 50% vs ByteBuffer on small columns like ints, even if we are comparing off-heap-bytebuffer with on-heap-byte[], simply because BB's extra fields are so big. That is, bq. A HeapByteBuffer wastes a lot of memory compared to a byte[] (5 more ints, a long, and a boolean). also goes for DirectByteBuffer (those fields are all declared in ByteBuffer that HBB and DBB extend). Of course, as the data size increases, this advantage disappears as DBB's heap usage stays constant while byte[] or HBB grows with it. was (Author: jbellis): I don't think switching to byte[] is crazy either. We'd save over 50% vs ByteBuffer on small columns like ints, even if we are comparing off-heap-bytebuffer with on-heap-byte[], simply because BB's extra fields are so big. That is, bq. A HeapByteBuffer wastes a lot of memory compared to a byte[] (5 more ints, a long, and a boolean). also goes for DirectByteBuffer (those fields are all declared in ByteBuffer that HBB and DBB extend). > Time to switch back to byte[] internally? > - > > Key: CASSANDRA-5020 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5020 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis > Fix For: 2.0 > > > We switched to ByteBuffer for column names and values back in 0.7, which gave > us a short term performance boost on mmap'd reads, but we gave that up when > we switched to refcounted sstables in 1.0. (refcounting all the way up the > read path would be too painful, so we copy into an on-heap buffer when > reading from an sstable, then release the reference.) > A HeapByteBuffer wastes a lot of memory compared to a byte[] (5 more ints, a > long, and a boolean). > The hard problem here is how to do the arena allocation we do on writes, > which has been very successful in reducing STW CMS from heap fragmentation. > ByteBuffer is a good fit there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5020) Time to switch back to byte[] internally?
[ https://issues.apache.org/jira/browse/CASSANDRA-5020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529052#comment-13529052 ] Jonathan Ellis edited comment on CASSANDRA-5020 at 12/11/12 3:28 PM: - I don't think switching to byte[] is crazy either. We'd save over 50% vs ByteBuffer on small columns like ints, even if we are comparing off-heap-bytebuffer with on-heap-byte[], simply because BB's extra fields are so big. That is, bq. A HeapByteBuffer wastes a lot of memory compared to a byte[] (5 more ints, a long, and a boolean). also goes for DirectByteBuffer (those fields are all declared in ByteBuffer that HBB and DBB extend). was (Author: jbellis): I don't think switching to byte[] is crazy either. We'd save over 50% vs ByteBuffer on small columns like ints, even if we are comparing off-heap-bytebuffer with on-heap-byte[], simply because BB's extra fields are so big. > Time to switch back to byte[] internally? > - > > Key: CASSANDRA-5020 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5020 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis > Fix For: 2.0 > > > We switched to ByteBuffer for column names and values back in 0.7, which gave > us a short term performance boost on mmap'd reads, but we gave that up when > we switched to refcounted sstables in 1.0. (refcounting all the way up the > read path would be too painful, so we copy into an on-heap buffer when > reading from an sstable, then release the reference.) > A HeapByteBuffer wastes a lot of memory compared to a byte[] (5 more ints, a > long, and a boolean). > The hard problem here is how to do the arena allocation we do on writes, > which has been very successful in reducing STW CMS from heap fragmentation. > ByteBuffer is a good fit there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5020) Time to switch back to byte[] internally?
[ https://issues.apache.org/jira/browse/CASSANDRA-5020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529052#comment-13529052 ] Jonathan Ellis commented on CASSANDRA-5020: --- I don't think switching to byte[] is crazy either. We'd save over 50% vs ByteBuffer on small columns like ints, even if we are comparing off-heap-bytebuffer with on-heap-byte[], simply because BB's extra fields are so big. > Time to switch back to byte[] internally? > - > > Key: CASSANDRA-5020 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5020 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis > Fix For: 2.0 > > > We switched to ByteBuffer for column names and values back in 0.7, which gave > us a short term performance boost on mmap'd reads, but we gave that up when > we switched to refcounted sstables in 1.0. (refcounting all the way up the > read path would be too painful, so we copy into an on-heap buffer when > reading from an sstable, then release the reference.) > A HeapByteBuffer wastes a lot of memory compared to a byte[] (5 more ints, a > long, and a boolean). > The hard problem here is how to do the arena allocation we do on writes, > which has been very successful in reducing STW CMS from heap fragmentation. > ByteBuffer is a good fit there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5020) Time to switch back to byte[] internally?
[ https://issues.apache.org/jira/browse/CASSANDRA-5020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529049#comment-13529049 ] Jonathan Ellis commented on CASSANDRA-5020: --- CF is sort of the right place for single-row reads but we need a new data structure for multi-row, currently we just use List. Then you need to special-case OutboundTCPConnection to clean those up. Or maybe introduce a MessageOut.cleanup method and subclass it for this. I say "sort of" because having a List or Set of sstables doesn't make sense for writes. Maybe it is time to split up "CF-for-writes" from "CF-for-reads." > Time to switch back to byte[] internally? > - > > Key: CASSANDRA-5020 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5020 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis > Fix For: 2.0 > > > We switched to ByteBuffer for column names and values back in 0.7, which gave > us a short term performance boost on mmap'd reads, but we gave that up when > we switched to refcounted sstables in 1.0. (refcounting all the way up the > read path would be too painful, so we copy into an on-heap buffer when > reading from an sstable, then release the reference.) > A HeapByteBuffer wastes a lot of memory compared to a byte[] (5 more ints, a > long, and a boolean). > The hard problem here is how to do the arena allocation we do on writes, > which has been very successful in reducing STW CMS from heap fragmentation. > ByteBuffer is a good fit there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529043#comment-13529043 ] Sylvain Lebresne commented on CASSANDRA-5043: - Yeah, if memory serves, what counters needed was to replace columns but to fix that we added a replace method directly instead of doing a remove+insert. > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: ABSC-with-addAll-for-time-ordered-data.patch, > CASSANDRA-5043.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5031) Add ssl support to binary protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529042#comment-13529042 ] Sylvain Lebresne commented on CASSANDRA-5031: - bq. It's used by SimpleClient.SecurePipelineFactory's constructor. Oh. My surprise was that it wasn't use on the server side too. But looking further, the answer seems to be that the server don't need the trustStore part. Can't we however just add a boolean 'isServer' to createSSLContext and skip the trustStore stuffs in that case? The rest is the same and this keyStore business feels out of place in the Server class. The rest lgtm. > Add ssl support to binary protocol > -- > > Key: CASSANDRA-5031 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5031 > Project: Cassandra > Issue Type: New Feature > Components: API >Reporter: Jonathan Ellis >Assignee: Jason Brown > Fix For: 1.2.1 > > Attachments: > 0001-CASSANDRA-5031-add-ssl-support-to-cql-binary-protoco.patch, > 0002-CASSANDRA-5031_round2.patch, 0003-CASSANDRA-5031-round3.patch > > > CASSANDRA-4239 added support in Thrift -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5020) Time to switch back to byte[] internally?
[ https://issues.apache.org/jira/browse/CASSANDRA-5020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529037#comment-13529037 ] T Jake Luciani commented on CASSANDRA-5020: --- Those options sound to me pretty non-deterministic still. It would be better if we could track the file descriptors used to fulfill a local read (possibly in the ColumnFamily class) and pass those back along with the result. > Time to switch back to byte[] internally? > - > > Key: CASSANDRA-5020 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5020 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis > Fix For: 2.0 > > > We switched to ByteBuffer for column names and values back in 0.7, which gave > us a short term performance boost on mmap'd reads, but we gave that up when > we switched to refcounted sstables in 1.0. (refcounting all the way up the > read path would be too painful, so we copy into an on-heap buffer when > reading from an sstable, then release the reference.) > A HeapByteBuffer wastes a lot of memory compared to a byte[] (5 more ints, a > long, and a boolean). > The hard problem here is how to do the arena allocation we do on writes, > which has been very successful in reducing STW CMS from heap fragmentation. > ByteBuffer is a good fit there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529030#comment-13529030 ] Jonathan Ellis edited comment on CASSANDRA-5043 at 12/11/12 3:06 PM: - I thought we took out the ABSC assert because counter merging will add things out of order. Edit: Nope, it's still there. Memory must be playing tricks on me. was (Author: jbellis): I thought we took out the ABSC assert because counter merging will add things out of order. > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: ABSC-with-addAll-for-time-ordered-data.patch, > CASSANDRA-5043.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529030#comment-13529030 ] Jonathan Ellis commented on CASSANDRA-5043: --- I thought we took out the ABSC assert because counter merging will add things out of order. > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: ABSC-with-addAll-for-time-ordered-data.patch, > CASSANDRA-5043.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5020) Time to switch back to byte[] internally?
[ https://issues.apache.org/jira/browse/CASSANDRA-5020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529026#comment-13529026 ] Jonathan Ellis commented on CASSANDRA-5020: --- using madvise/mincore is race prone (we purge it from cache on compaction, a read in progress pulls some back into cache, now we wait indefinitely for it to get pushed out of cache by other data). using weakreferences means we're back to pre-1.0 of waiting for JVM GC before we can delete obsolete sstables. > Time to switch back to byte[] internally? > - > > Key: CASSANDRA-5020 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5020 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis > Fix For: 2.0 > > > We switched to ByteBuffer for column names and values back in 0.7, which gave > us a short term performance boost on mmap'd reads, but we gave that up when > we switched to refcounted sstables in 1.0. (refcounting all the way up the > read path would be too painful, so we copy into an on-heap buffer when > reading from an sstable, then release the reference.) > A HeapByteBuffer wastes a lot of memory compared to a byte[] (5 more ints, a > long, and a boolean). > The hard problem here is how to do the arena allocation we do on writes, > which has been very successful in reducing STW CMS from heap fragmentation. > ByteBuffer is a good fit there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5031) Add ssl support to binary protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-5031: --- Attachment: 0003-CASSANDRA-5031-round3.patch bq. Why don't we reuse the SSLFactory.createSSLContext method? It's used by SimpleClient.SecurePipelineFactory's constructor. bq. not sure I understand this SslStateHandler Looking at this code fresh, I think I got a little off-track when I ran into the reconnect bug (which I mentioned above). Reading the netty source, I see now that calling handshake() explicitly is, in fact, unessecary as SslHandler.channelConnected() will do that if issueHandshake is already set to true (which we are already doing). I, too, was unhappy with the pipeline duplication, but if we did need to inject a channel, copying and putting a big note seemed better than trying to dance around the ordering with putBefore(), putAfter(), and so on. Either way, we don't need the extra SslStateHandler (client and server sides), and it's removed in the attached patch. > Add ssl support to binary protocol > -- > > Key: CASSANDRA-5031 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5031 > Project: Cassandra > Issue Type: New Feature > Components: API >Reporter: Jonathan Ellis >Assignee: Jason Brown > Fix For: 1.2.1 > > Attachments: > 0001-CASSANDRA-5031-add-ssl-support-to-cql-binary-protoco.patch, > 0002-CASSANDRA-5031_round2.patch, 0003-CASSANDRA-5031-round3.patch > > > CASSANDRA-4239 added support in Thrift -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5031) Add ssl support to binary protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528977#comment-13528977 ] Sylvain Lebresne commented on CASSANDRA-5031: - Hum, that version add a bunch of stuff I'm not sure I understand: * Why don't we reuse the SSLFactory.createSSLContext method? The patch makes it public so I suspect that was the intention but the patch doesn't do it. * This may be linked to the reconnection problems you mention, but not sure I understand this SslStateHandler: ** First, I'm pretty sure you don't need to call handshake() especially if SslHandler.setIssueHandsake(true) has been called. And reading the javadoc of SSLEngine.beginHandshake (which handshake() calls) suggests that you don't even need to call it at all unless you do renogetiation. The only other reason to call handshake in Netty that I know of would be to know when the handshake is done, but I don't think we care about that here because we won't write anything on the wire anyway until everything is ready anyway. ** Then even if we happen to call handshake for some reason that I miss, I'm not sure why we put the channel in a ChannelGroup that as far as I can tell we don't use at all. ** I'm also unclear on why this handler should be between the message decoders and the dispatcher. If we really need to do something ssl related when a channel is connected (which again I'm not sure we do, though I could be wrong), why not just extends SslHandler and override it's channelConnected method? In particular we'll avoid that pipeline duplication (that I'd really rather avoid, including for the client side). bq. Should I avoid doing that for existing classes as that makes the diff noisier? In theory, we do try to avoid it if it's too much noise (which is subjective), and prefer leaving code style fixes to there own separate patches. Not a big deal though and if it doesn't hide too much the actual fixes it's fine. > Add ssl support to binary protocol > -- > > Key: CASSANDRA-5031 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5031 > Project: Cassandra > Issue Type: New Feature > Components: API >Reporter: Jonathan Ellis >Assignee: Jason Brown > Fix For: 1.2.1 > > Attachments: > 0001-CASSANDRA-5031-add-ssl-support-to-cql-binary-protoco.patch, > 0002-CASSANDRA-5031_round2.patch > > > CASSANDRA-4239 added support in Thrift -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5054) new partitioner for rowkey pairs (P1, P2) targeting hierarchical data
[ https://issues.apache.org/jira/browse/CASSANDRA-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528973#comment-13528973 ] Dominique De Vito commented on CASSANDRA-5054: -- Let's analyze the number of data communicated between Cassandra nodes, for RandomPartitioner and RandomPartitioner4Pair (the new submitted partitioner). The test is the following: * we read/write columns associated with rowkeys like (P1, _) where P1 is fixed/constant and the second element of the pair is changing. * N is the number of nodes of the cluster * L is the number of lines read or written for a given replica, for the test * all cluster nodes are used as (possible) coordinators. * RF=3 We evaluate the exchanges' cost between the coordinator and other nodes, to determine the reponse's cost for each partitioner. * RandomPartitioner Hypothesis: each node stores 1/N of all the lines, and the read or write lines (of the tests) are equally distributed within the cluster. So, the coordinator stores itself L / N lines among those used for responding to the request. The number of communicated lines between the coordinator and other nodes is: 2 * L/N + 3 * (L – L/N) The number of communications between the coordinator and other nodes is: N-1 * RandomPartitioner4Pair (new partitioner) As rowkeys requested are like (P1, _), with P1 fixed, all rowkeys are expected to be stored on 3 nodes only (due to RF=3). 2 cases: 1) the coordinator is one of the 3 nodes storing the requested lines 2) the other case For lines: 1) the number of communicated lines between the coordinator and other nodes is: 2 * L 2) the number of communicated lines between the coordinator and other nodes is: 3 * L In average: (3 * 2 * L + (N-3) * 3 * L) / N For communications: 1) the number of communications between the coordinator and other nodes is: 2 2) the number of communications between the coordinator and other nodes is: 3 In average: (3 * 2 + (N-3) * 3) / N * Conclusion (1/2) - For the number of communicated lines between the coordinator and other nodes: both partitioners are roughly quite equal. RandomPartitioner4Pair: (3 * 2 * L + (N-3) * 3 * L) / N = 3 L * (N-1) / N = L * (3 N – 3) / N RandomPartitioner: 2 * L/N + 3 * (L – L/N) = 3 * L – L / N = L * (3 – 1 / N) = L * (3 N – 1) / N In fact, RandomPartitioner4Pair has a little advantage over RandomPartitioner : cost(RandomPartitioner4Pair) < cost(RandomPartitioner). Here, all cluster nodes are used as (possible) coordinators. So, the calculated costs above are average costs. So, in average, RandomPartitioner4Pair (the submitted new partitioner) is a little better than RandomPartitioner. But, if we use Astyanax, it's possible to choose the coordinator among the replica. And in that case, the cost of using RandomPartitioner4Pair is then much better than RandomPartitioner. - For the number of communications between the coordinator and other nodes: RandomPartitioner4Pair has an advantage. * Conclusion (2/2) RandomPartitioner4Pair (the submitted new partitioner) is a little better than RandomPartitioner in average, or much more better than RandomPartitioner when used with Astyanax, when choosing the coordinator among the replica. > new partitioner for rowkey pairs (P1, P2) targeting hierarchical data > - > > Key: CASSANDRA-5054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5054 > Project: Cassandra > Issue Type: Improvement > Components: Contrib, Core >Reporter: Dominique De Vito >Priority: Trivial > Attachments: RandomPartitioner4Pair.java > > > This new partitioner is submitted here both for validation and for proposal > to other Cassandra users (may be this partitioner has its place within > Cassandra core, or into some 'contrib' directory). > This new partitioner is a variant of RandomPartitioner with special token > computation for composite rowkeys of size 2. > The use case of this partitioner is about rowkeys of hierarchical data, that > is, rowkeys like (directory id, file id). > This partitioner computes tokens so that rows with same "directory id" have a > great chance to be on the same node: the goal is that, when Cassandra is > asked about multiple file ids for the same directory id, a limited number of > nodes should be asked. > So, in case of a composite rowkey of size 2 like (P1, P2), the partitioner > computes the token as follows: merge(getHighBits(md5(P1)), > getLowBits(md5(P2))). > In case of a rowkey that is NOT a pair, the partitioner returns the same > value than RandomPartitioner. > This partitioner is expected to be used with Cassandra 1.1 or above. > Cassandra stores in sstables the pair (token, rowkey). Since v1.1, during a > compaction phas
[jira] [Updated] (CASSANDRA-5054) new partitioner for rowkey pairs (P1, P2) targeting hierarchical data
[ https://issues.apache.org/jira/browse/CASSANDRA-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominique De Vito updated CASSANDRA-5054: - Attachment: RandomPartitioner4Pair.java the new partitioner code > new partitioner for rowkey pairs (P1, P2) targeting hierarchical data > - > > Key: CASSANDRA-5054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5054 > Project: Cassandra > Issue Type: Improvement > Components: Contrib, Core >Reporter: Dominique De Vito >Priority: Trivial > Attachments: RandomPartitioner4Pair.java > > > This new partitioner is submitted here both for validation and for proposal > to other Cassandra users (may be this partitioner has its place within > Cassandra core, or into some 'contrib' directory). > This new partitioner is a variant of RandomPartitioner with special token > computation for composite rowkeys of size 2. > The use case of this partitioner is about rowkeys of hierarchical data, that > is, rowkeys like (directory id, file id). > This partitioner computes tokens so that rows with same "directory id" have a > great chance to be on the same node: the goal is that, when Cassandra is > asked about multiple file ids for the same directory id, a limited number of > nodes should be asked. > So, in case of a composite rowkey of size 2 like (P1, P2), the partitioner > computes the token as follows: merge(getHighBits(md5(P1)), > getLowBits(md5(P2))). > In case of a rowkey that is NOT a pair, the partitioner returns the same > value than RandomPartitioner. > This partitioner is expected to be used with Cassandra 1.1 or above. > Cassandra stores in sstables the pair (token, rowkey). Since v1.1, during a > compaction phase, Cassandra does not sort anymore on token only, but > sort on the pair (token, rowkey). So, if a custom partitioner produces token > collisions (that is, multiple rowkeys with the same token), it won't be a pb > with v1.1 because, in order to retrieve a row, Cassandra is going to compare > the full pair (token, rowkey). > As the percent of row collisions for this partitioner is unknown, it's > expected to be used with Cassandra 1.1 or above. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5054) new partitioner for rowkey pairs (P1, P2) targeting hierarchical data
[ https://issues.apache.org/jira/browse/CASSANDRA-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominique De Vito updated CASSANDRA-5054: - Description: This new partitioner is submitted here both for validation and for proposal to other Cassandra users (may be this partitioner has its place within Cassandra core, or into some 'contrib' directory). This new partitioner is a variant of RandomPartitioner with special token computation for composite rowkeys of size 2. The use case of this partitioner is about rowkeys of hierarchical data, that is, rowkeys like (directory id, file id). This partitioner computes tokens so that rows with same "directory id" have a great chance to be on the same node: the goal is that, when Cassandra is asked about multiple file ids for the same directory id, a limited number of nodes should be asked. So, in case of a composite rowkey of size 2 like (P1, P2), the partitioner computes the token as follows: merge(getHighBits(md5(P1)), getLowBits(md5(P2))). In case of a rowkey that is NOT a pair, the partitioner returns the same value than RandomPartitioner. This partitioner is expected to be used with Cassandra 1.1 or above. Cassandra stores in sstables the pair (token, rowkey). Since v1.1, during a compaction phase, Cassandra does not sort anymore on token only, but sort on the pair (token, rowkey). So, if a custom partitioner produces token collisions (that is, multiple rowkeys with the same token), it won't be a pb with v1.1 because, in order to retrieve a row, Cassandra is going to compare the full pair (token, rowkey). As the percent of row collisions for this partitioner is unknown, it's expected to be used with Cassandra 1.1 or above. > new partitioner for rowkey pairs (P1, P2) targeting hierarchical data > - > > Key: CASSANDRA-5054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5054 > Project: Cassandra > Issue Type: Improvement > Components: Contrib, Core >Reporter: Dominique De Vito >Priority: Trivial > > This new partitioner is submitted here both for validation and for proposal > to other Cassandra users (may be this partitioner has its place within > Cassandra core, or into some 'contrib' directory). > This new partitioner is a variant of RandomPartitioner with special token > computation for composite rowkeys of size 2. > The use case of this partitioner is about rowkeys of hierarchical data, that > is, rowkeys like (directory id, file id). > This partitioner computes tokens so that rows with same "directory id" have a > great chance to be on the same node: the goal is that, when Cassandra is > asked about multiple file ids for the same directory id, a limited number of > nodes should be asked. > So, in case of a composite rowkey of size 2 like (P1, P2), the partitioner > computes the token as follows: merge(getHighBits(md5(P1)), > getLowBits(md5(P2))). > In case of a rowkey that is NOT a pair, the partitioner returns the same > value than RandomPartitioner. > This partitioner is expected to be used with Cassandra 1.1 or above. > Cassandra stores in sstables the pair (token, rowkey). Since v1.1, during a > compaction phase, Cassandra does not sort anymore on token only, but > sort on the pair (token, rowkey). So, if a custom partitioner produces token > collisions (that is, multiple rowkeys with the same token), it won't be a pb > with v1.1 because, in order to retrieve a row, Cassandra is going to compare > the full pair (token, rowkey). > As the percent of row collisions for this partitioner is unknown, it's > expected to be used with Cassandra 1.1 or above. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5054) new partitioner for rowkey pairs (P1, P2) targeting hierarchical data
Dominique De Vito created CASSANDRA-5054: Summary: new partitioner for rowkey pairs (P1, P2) targeting hierarchical data Key: CASSANDRA-5054 URL: https://issues.apache.org/jira/browse/CASSANDRA-5054 Project: Cassandra Issue Type: Improvement Components: Contrib, Core Reporter: Dominique De Vito Priority: Trivial -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5053) not possible to change crc_check_chance
[ https://issues.apache.org/jira/browse/CASSANDRA-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-5053: --- Attachment: 0001-fix-CASSANDRA-5053-not-possible-to-change-crc_check_.patch > not possible to change crc_check_chance > --- > > Key: CASSANDRA-5053 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5053 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.1, 1.1.7 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.1.7 > > Attachments: > 0001-fix-CASSANDRA-5053-not-possible-to-change-crc_check_.patch > > > It is not possible to change crc_check_chance using a schema modification > after CASSANDRA-4266 > This patch fixes that and moves the setting out into a configuration > parameter instead, you dont want to upgrade/scrub/.. all your sstables to > change the crc_check_chance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5053) not possible to change crc_check_chance
Marcus Eriksson created CASSANDRA-5053: -- Summary: not possible to change crc_check_chance Key: CASSANDRA-5053 URL: https://issues.apache.org/jira/browse/CASSANDRA-5053 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.7, 1.1.1 Reporter: Marcus Eriksson Assignee: Marcus Eriksson Priority: Minor Fix For: 1.1.7 It is not possible to change crc_check_chance using a schema modification after CASSANDRA-4266 This patch fixes that and moves the setting out into a configuration parameter instead, you dont want to upgrade/scrub/.. all your sstables to change the crc_check_chance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528867#comment-13528867 ] Sylvain Lebresne commented on CASSANDRA-5043: - Yes, that's what I was suggesting (except that the patch doesn't seem to apply to cassandra-1.2.0, that I don't think we should use temp in the reduceNameFilter call and maybe we've want to optimize for when there is only one source of columns (but maybe it's not worth bothering)). > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: ABSC-with-addAll-for-time-ordered-data.patch, > CASSANDRA-5043.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Git Push Summary
Updated Tags: refs/tags/1.2.0-rc1-tentative [created] d791e0b3f
[jira] [Updated] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-5043: --- Attachment: ABSC-with-addAll-for-time-ordered-data.patch Is this want you mean? first populate temp then put it in bulk to ABSC using addAll? > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: ABSC-with-addAll-for-time-ordered-data.patch, > CASSANDRA-5043.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528816#comment-13528816 ] Sylvain Lebresne commented on CASSANDRA-5043: - bq. ABSC still looks better than TMBSC in practice. Again, I'm not contesting that! But saying it 10 times won't change the fact that ABSC currently throw an assertion if you insert columns out of order, and that columnTimeOrderedData does that kind of out-of-order insertions. As for the performance part, I was not necessarily in favor of adding the assertion in the first place for that very reason, and yes, because this is for names queries, and a such the number of columns is unlikely to be very large, ABSC (without the assertion) would almost surely be always faster than TMBSC. Though as said in my first comment, in term of complexity, aggregating all the result of each sstable (so all in order) in a first ABSC first and then using addAll() to merge it to a result ABSC would be better (and that latter solution can be done without even removing the ABSC assertion). > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: CASSANDRA-5043.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528810#comment-13528810 ] Pavel Yaskevich commented on CASSANDRA-5043: I'm saying this because there a particular time-ordered workload involving multiple sstables which shouldn't be all sorted and if assertion is not firing it could mean that something is broken or assertions are disabled, I will double check that tomorrow. > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: CASSANDRA-5043.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5052) cassandra-cli should escape keyspace name
Leonid Shalupov created CASSANDRA-5052: -- Summary: cassandra-cli should escape keyspace name Key: CASSANDRA-5052 URL: https://issues.apache.org/jira/browse/CASSANDRA-5052 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.7 Reporter: Leonid Shalupov show schema yields "use __someKeyspace", expecting "use '__someKeyspace'". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528804#comment-13528804 ] Pavel Yaskevich commented on CASSANDRA-5043: Actually, I will check tomorrow if the cluster we use are running with assertions disabled even if that is the case ABSC still looks better than TMBSC in practice. > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: CASSANDRA-5043.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528803#comment-13528803 ] Sylvain Lebresne commented on CASSANDRA-5043: - I've shown you already: CASSANDRA-3289. > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: CASSANDRA-5043.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528793#comment-13528793 ] Pavel Yaskevich commented on CASSANDRA-5043: Show me, never got a single one. > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: CASSANDRA-5043.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: Update version for 1.2.0-rc1 release
Updated Branches: refs/heads/cassandra-1.2.0 a399a5644 -> d791e0b3f Update version for 1.2.0-rc1 release Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d791e0b3 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d791e0b3 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d791e0b3 Branch: refs/heads/cassandra-1.2.0 Commit: d791e0b3fa5a615f2df200ecd40f82dc9a5874d6 Parents: a399a56 Author: Sylvain Lebresne Authored: Tue Dec 11 09:10:45 2012 +0100 Committer: Sylvain Lebresne Committed: Tue Dec 11 09:10:45 2012 +0100 -- build.xml|2 +- debian/changelog |6 ++ 2 files changed, 7 insertions(+), 1 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d791e0b3/build.xml -- diff --git a/build.xml b/build.xml index 1f75261..e9b8e78 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> http://git-wip-us.apache.org/repos/asf/cassandra/blob/d791e0b3/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index 3bb10aa..cfa1781 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (1.2.0~rc1) unstable; urgency=low + + * New RC release + + -- Sylvain Lebresne Tue, 11 Dec 2012 09:09:08 +0100 + cassandra (1.2.0~beta3) unstable; urgency=low * New beta release
[jira] [Commented] (CASSANDRA-5043) Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) fix.
[ https://issues.apache.org/jira/browse/CASSANDRA-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528786#comment-13528786 ] Sylvain Lebresne commented on CASSANDRA-5043: - I'm not arguing performance, I'm saying the attached patch is buggy and will throw assertion errors. > Small CollationController refactoring and CFS.getRawCachedRow(DecoratedKey) > fix. > > > Key: CASSANDRA-5043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5043 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.2.0 > > Attachments: CASSANDRA-5043.patch > > > CollactionController can construct a factory on init based on mutableColumns > parameter. ArrayBackedSortedColumns proven to be better for counters as well > as normal columns (row size from 5 to 75 columns). CFS.getRawCachedRow would > copy key byte array each time it contracts a RowCacheKey so we want to avoid > that where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
