[jira] [Updated] (CASSANDRA-2394) Faulty hd kills cluster performance
[ https://issues.apache.org/jira/browse/CASSANDRA-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thibaut updated CASSANDRA-2394: --- Fix Version/s: 0.7.5 Faulty hd kills cluster performance --- Key: CASSANDRA-2394 URL: https://issues.apache.org/jira/browse/CASSANDRA-2394 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.4 Reporter: Thibaut Priority: Minor Fix For: 0.7.5 Hi, About every week, a node from our main cluster (100 nodes) has a faulty hd (Listing the cassandra data storage directoy triggers an input/output error). Whenever this occurs, I see many timeoutexceptions in our application on various nodes which cause everything to run very very slowly. Keyrange scans just timeout and will sometimes never succeed. If I stop cassandra on the faulty node, everything runs normal again. It would be great to have some kind of monitoring thread in cassandra which marks a node as down if there are multiple read/write errors to the data directories. A single faulty hd on 1 node shouldn't affect global cluster performance. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2395) Seems daylight saving time changes are not handled correctly in system.log
Seems daylight saving time changes are not handled correctly in system.log -- Key: CASSANDRA-2395 URL: https://issues.apache.org/jira/browse/CASSANDRA-2395 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.4 Environment: java version 1.6.0_23 Java(TM) SE Runtime Environment (build 1.6.0_23-b05) Java HotSpot(TM) Server VM (build 19.0-b09, mixed mode) Linux 2.6.37-5 #1 SMP Thu Feb 10 20:20:17 CET 2011 x86_64 AMD_Phenom(tm)_II_X6_1055T_Processor PLD Linux Reporter: Mateusz Korniak Priority: Minor Cassandra running during daylight saving time change does not reflect it in logs, still displaying an old (winter in may case) time: INFO [AntiEntropyStage:1] 2011-03-28 12:04:21,576 while date returns: Mon Mar 28 13:04:26 CEST 2011 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2324) Repair transfers more data than necessary
[
https://issues.apache.org/jira/browse/CASSANDRA-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012034#comment-13012034
]
Sylvain Lebresne commented on CASSANDRA-2324:
-
The problem is, the ranges repair hashes are not actual node ranges.
Let's consider the following ring (RF=2), where I consider token being in
[0..12] to simplify, and where everything is consistent:
{noformat}
_.--._
C (token: 11) .' `.
[11,3][3,7] / \
||
|| A(token: 3)
|| [3,7],[7,11]
\ /
`.__.'
B (token: 7)`--'
[7,11],[11,3]
{noformat}
Now say I run a repair on node A. The problem is that the Merkle tree ranges
are built by dividing the full range by 2 recursively. This means that in this
example, the ranges in the tree will for instance be [0,2], [2, 4], [4, 6], [6,
8], [8,10] and [10,12].
If you compare the hashes for A and B on those ranges, changes are you'll find
mismatches for [6,8] and [10,12] (because A don't have anyone on [11, 12] while
B have, and B don't have anyone on [6, 7] while A have). As a consequence, the
range [7,8] and [10,11] will be repaired, even though there is no
inconsistencies.
What that means in practice is that it will be very rare for anti-antropy to
actually consider the nodes in sync, it will almost surely repair something,
even if the nodes are perfectly consistent. It's Very easy to check btw: with a
cluster right the one above (3 nodes, RF=2), with as few as 5 keys for the
whole cluster I'm able to have a repair do repairs over and over again.
Now the good question is: how bad is it ? I'm not sure, I depends a bit.
On a 3 nodes cluster (RF=2), I tried inserting 1M keys with stress (stress -l
2) and triggered repair afterwards. The amount of (unnecessarily) repaired keys
was around 150 keys for a given node (it varies slightly for run to run because
there is some randomness in the creation of the Merkle tree), corresponding to
~44KB streamed (that is the amount transfered to the node where repair has been
ran, so for the total operation its twice this, since we stream in both ways).
That's ~0.02% of keys (a given node have ~666 666 keys). It's bad to do
useless work, but not a really big deal.
However, the less keys we'll have, the worst it gets (and the bigger our rows
are, the more useless transfer we do). With the same experiment inserting only
10K keys, there is 190 keys uselessly repaired. That's now close to 3% of the
load. It also gets worst with increasing replication factor.
To fix this, we would need for the range in the Merkle tree to share the node
range boundaries. An interesting way to do this would be to have the
coordinating node give a list a range for which to calculate Merkle trees, and
the node would compute one tree by range (for the coordinating node, that would
be #RF's tree). A nice think with this is that it would leave room to
optimizing repair since a node would need to do a validation compaction only on
the range asked for, which means that only the coordinator node would validate
all its data. The neighbors would do less work.
Repair transfers more data than necessary
-
Key: CASSANDRA-2324
URL: https://issues.apache.org/jira/browse/CASSANDRA-2324
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Sylvain Lebresne
Fix For: 0.7.5
To repro: 3 node cluster, stress.java 1M rows with -x KEYS and -l 2. The
index is enough to make some mutations drop (about 20-30k total in my tests).
Repair afterwards will repair a large amount of ranges the first time.
However, each subsequent run will repair the same set of small ranges every
time. INDEXED_RANGE_SLICE in stress never fully works. Counting rows with
sstablekeys shows there are 2M rows total as expected, however when trying to
count the indexed keys, I get exceptions like:
{noformat}
Exception in thread main java.io.IOException: Key out of order!
DecoratedKey(101571366040797913119296586470838356016,
0707ab782c5b5029d28a5e6d508ef72f0222528b5e28da3b7787492679dc51b96f868e0746073e54bc173be927049d0f51e25a6a95b3268213b8969abf40cea7d7)
DecoratedKey(12639574763031545147067490818595764132,
0bc414be3093348a2ad389ed28f18f0cc9a044b2e98587848a0d289dae13ed0ad479c74654900eeffc6236)
at
org.apache.cassandra.tools.SSTableExport.enumeratekeys(SSTableExport.java:206)
at
org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:388)
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see:
[jira] [Commented] (CASSANDRA-2381) orphaned data files may be created during migration race
[ https://issues.apache.org/jira/browse/CASSANDRA-2381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012045#comment-13012045 ] Gary Dusbabek commented on CASSANDRA-2381: -- Is it necessary to make the MT lock static? I do not think there would be a problem with concurrently flushing two memtables of different column families (the CL serialization is still preserved by the synchronization in CFS). orphaned data files may be created during migration race Key: CASSANDRA-2381 URL: https://issues.apache.org/jira/browse/CASSANDRA-2381 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.5 Attachments: 2381-0.8.txt We try to prevent creating orphans by locking Table.flusherLock in maybeSwitchMemtable and the Migration process, but since the actual writing is done asynchronously in Memtable.writeSortedContents there is a race window, where we acquire lock in maybeSwitch, we're not dropped so we queue the flush and release the lock, Migration does the drop, then Memtable writes itself out. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure of cassandra-trunk on ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1180 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: Build Source Stamp: [branch cassandra/trunk] 1086226 Blamelist: slebresne BUILD FAILED: failed compile sincerely, -The Buildbot
[jira] [Resolved] (CASSANDRA-2395) Seems daylight saving time changes are not handled correctly in system.log
[ https://issues.apache.org/jira/browse/CASSANDRA-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2395. --- Resolution: Won't Fix Not a Cassandra problem. Seems daylight saving time changes are not handled correctly in system.log -- Key: CASSANDRA-2395 URL: https://issues.apache.org/jira/browse/CASSANDRA-2395 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.4 Environment: java version 1.6.0_23 Java(TM) SE Runtime Environment (build 1.6.0_23-b05) Java HotSpot(TM) Server VM (build 19.0-b09, mixed mode) Linux 2.6.37-5 #1 SMP Thu Feb 10 20:20:17 CET 2011 x86_64 AMD_Phenom(tm)_II_X6_1055T_Processor PLD Linux Reporter: Mateusz Korniak Priority: Minor Cassandra running during daylight saving time change does not reflect it in logs, still displaying an old (winter in may case) time: INFO [AntiEntropyStage:1] 2011-03-28 12:04:21,576 while date returns: Mon Mar 28 13:04:26 CEST 2011 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2324) Repair transfers more data than necessary
[
https://issues.apache.org/jira/browse/CASSANDRA-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012056#comment-13012056
]
Jonathan Ellis commented on CASSANDRA-2324:
---
bq. To fix this, we would need for the range in the Merkle tree to share the
node range boundaries
couldn't we just take the interesection of the computed ranges w/ the range
actually being repaired?
Repair transfers more data than necessary
-
Key: CASSANDRA-2324
URL: https://issues.apache.org/jira/browse/CASSANDRA-2324
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Sylvain Lebresne
Fix For: 0.7.5
To repro: 3 node cluster, stress.java 1M rows with -x KEYS and -l 2. The
index is enough to make some mutations drop (about 20-30k total in my tests).
Repair afterwards will repair a large amount of ranges the first time.
However, each subsequent run will repair the same set of small ranges every
time. INDEXED_RANGE_SLICE in stress never fully works. Counting rows with
sstablekeys shows there are 2M rows total as expected, however when trying to
count the indexed keys, I get exceptions like:
{noformat}
Exception in thread main java.io.IOException: Key out of order!
DecoratedKey(101571366040797913119296586470838356016,
0707ab782c5b5029d28a5e6d508ef72f0222528b5e28da3b7787492679dc51b96f868e0746073e54bc173be927049d0f51e25a6a95b3268213b8969abf40cea7d7)
DecoratedKey(12639574763031545147067490818595764132,
0bc414be3093348a2ad389ed28f18f0cc9a044b2e98587848a0d289dae13ed0ad479c74654900eeffc6236)
at
org.apache.cassandra.tools.SSTableExport.enumeratekeys(SSTableExport.java:206)
at
org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:388)
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2394) Faulty hd kills cluster performance
[ https://issues.apache.org/jira/browse/CASSANDRA-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012057#comment-13012057 ] Gary Dusbabek commented on CASSANDRA-2394: -- This probably belongs in 0.8. Faulty hd kills cluster performance --- Key: CASSANDRA-2394 URL: https://issues.apache.org/jira/browse/CASSANDRA-2394 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.4 Reporter: Thibaut Priority: Minor Fix For: 0.7.5 Hi, About every week, a node from our main cluster (100 nodes) has a faulty hd (Listing the cassandra data storage directoy triggers an input/output error). Whenever this occurs, I see many timeoutexceptions in our application on various nodes which cause everything to run very very slowly. Keyrange scans just timeout and will sometimes never succeed. If I stop cassandra on the faulty node, everything runs normal again. It would be great to have some kind of monitoring thread in cassandra which marks a node as down if there are multiple read/write errors to the data directories. A single faulty hd on 1 node shouldn't affect global cluster performance. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build of cassandra-trunk on ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1181 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: Build Source Stamp: [branch cassandra/trunk] 1086232 Blamelist: slebresne Build succeeded! sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-2324) Repair transfers more data than necessary
[
https://issues.apache.org/jira/browse/CASSANDRA-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012062#comment-13012062
]
Sylvain Lebresne commented on CASSANDRA-2324:
-
bq. couldn't we just take the interesection of the computed ranges w/ the range
actually being repaired?
We do that. But the problem is: you're node A and you receive a merkle tree
from B that in particular says that for the range [0..10] the hash is x. And on
[0..10] your has is x'. The problem is when [0..10] is partly one of your
range, partly not. For instance it can be that you're a replica for [8..10] but
not at all for [0..8].
This is due to the fact that the ranges for which the hashes are computed are
computed without concern for actual node ranges. So now you know there is some
inconsistency on [0..10] but it may just be that B is responsible for [0..8]
and have data for it (and we don't since we are not in charge of that).
In that case, the code do take the intersection of [0..10] with the local range
and will stream only [8..10]. But it's still useless.
Repair transfers more data than necessary
-
Key: CASSANDRA-2324
URL: https://issues.apache.org/jira/browse/CASSANDRA-2324
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Sylvain Lebresne
Fix For: 0.7.5
To repro: 3 node cluster, stress.java 1M rows with -x KEYS and -l 2. The
index is enough to make some mutations drop (about 20-30k total in my tests).
Repair afterwards will repair a large amount of ranges the first time.
However, each subsequent run will repair the same set of small ranges every
time. INDEXED_RANGE_SLICE in stress never fully works. Counting rows with
sstablekeys shows there are 2M rows total as expected, however when trying to
count the indexed keys, I get exceptions like:
{noformat}
Exception in thread main java.io.IOException: Key out of order!
DecoratedKey(101571366040797913119296586470838356016,
0707ab782c5b5029d28a5e6d508ef72f0222528b5e28da3b7787492679dc51b96f868e0746073e54bc173be927049d0f51e25a6a95b3268213b8969abf40cea7d7)
DecoratedKey(12639574763031545147067490818595764132,
0bc414be3093348a2ad389ed28f18f0cc9a044b2e98587848a0d289dae13ed0ad479c74654900eeffc6236)
at
org.apache.cassandra.tools.SSTableExport.enumeratekeys(SSTableExport.java:206)
at
org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:388)
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2381) orphaned data files may be created during migration race
[ https://issues.apache.org/jira/browse/CASSANDRA-2381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2381: -- Attachment: 2831-v2.txt You're right. v2 attached and rebased. Had to make lock/unlock responsibility of individual Migration classes since lock is no longer global. orphaned data files may be created during migration race Key: CASSANDRA-2381 URL: https://issues.apache.org/jira/browse/CASSANDRA-2381 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.5 Attachments: 2381-0.8.txt, 2831-v2.txt We try to prevent creating orphans by locking Table.flusherLock in maybeSwitchMemtable and the Migration process, but since the actual writing is done asynchronously in Memtable.writeSortedContents there is a race window, where we acquire lock in maybeSwitch, we're not dropped so we queue the flush and release the lock, Migration does the drop, then Memtable writes itself out. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2324) Repair transfers more data than necessary
[
https://issues.apache.org/jira/browse/CASSANDRA-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012071#comment-13012071
]
Jonathan Ellis commented on CASSANDRA-2324:
---
I thought repair is per-token-range, i.e., if I say nodetool repair A then
range (11, 3] and (3, 7] will be repaired independently.
Repair transfers more data than necessary
-
Key: CASSANDRA-2324
URL: https://issues.apache.org/jira/browse/CASSANDRA-2324
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Sylvain Lebresne
Fix For: 0.7.5
To repro: 3 node cluster, stress.java 1M rows with -x KEYS and -l 2. The
index is enough to make some mutations drop (about 20-30k total in my tests).
Repair afterwards will repair a large amount of ranges the first time.
However, each subsequent run will repair the same set of small ranges every
time. INDEXED_RANGE_SLICE in stress never fully works. Counting rows with
sstablekeys shows there are 2M rows total as expected, however when trying to
count the indexed keys, I get exceptions like:
{noformat}
Exception in thread main java.io.IOException: Key out of order!
DecoratedKey(101571366040797913119296586470838356016,
0707ab782c5b5029d28a5e6d508ef72f0222528b5e28da3b7787492679dc51b96f868e0746073e54bc173be927049d0f51e25a6a95b3268213b8969abf40cea7d7)
DecoratedKey(12639574763031545147067490818595764132,
0bc414be3093348a2ad389ed28f18f0cc9a044b2e98587848a0d289dae13ed0ad479c74654900eeffc6236)
at
org.apache.cassandra.tools.SSTableExport.enumeratekeys(SSTableExport.java:206)
at
org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:388)
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2373) Index Out Of Bounds during Validation Compaction (Repair)
[ https://issues.apache.org/jira/browse/CASSANDRA-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2373: -- Fix Version/s: 0.7.5 Assignee: Sylvain Lebresne Index Out Of Bounds during Validation Compaction (Repair) - Key: CASSANDRA-2373 URL: https://issues.apache.org/jira/browse/CASSANDRA-2373 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.4 Environment: CentOS on EC2 Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Fix For: 0.7.5 Stack Trace is below: ERROR [CompactionExecutor:1] 2011-03-23 19:11:39,488 AbstractCassandraDaemon.java (line 112) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.lang.IndexOutOfBoundsException at java.nio.Buffer.checkIndex(Unknown Source) at java.nio.HeapByteBuffer.getInt(Unknown Source) at org.apache.cassandra.db.DeletedColumn.getLocalDeletionTime(DeletedColumn.java:57) at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:879) at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedColumnsOnly(ColumnFamilyStore.java:866) at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:857) at org.apache.cassandra.io.PrecompactedRow.init(PrecompactedRow.java:94) at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:147) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94) at org.apache.cassandra.db.CompactionManager.doValidationCompaction(CompactionManager.java:822) at org.apache.cassandra.db.CompactionManager.access$800(CompactionManager.java:56) at org.apache.cassandra.db.CompactionManager$6.call(CompactionManager.java:358) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2284) Make changes to the set of memtables and sstables of a cfstore atomic
[ https://issues.apache.org/jira/browse/CASSANDRA-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012080#comment-13012080 ] Hudson commented on CASSANDRA-2284: --- Integrated in Cassandra #810 (See [https://hudson.apache.org/hudson/job/Cassandra/810/]) Fix build (for CASSANDRA-2284) Atomically switch cfstore memtables and sstables patch by slebresne; reviewed by jbellis for CASSANDRA-2284 (and CASSANDRA-2105) Make changes to the set of memtables and sstables of a cfstore atomic -- Key: CASSANDRA-2284 URL: https://issues.apache.org/jira/browse/CASSANDRA-2284 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.8 Attachments: 0001-Make-memtable-and-sstable-switches-atomic-v2.patch, 0001-Make-memtable-and-sstable-switches-atomic-v3.patch, 0001-Make-memtable-and-sstable-switches-atomic.patch, 2284-v2-rebased.txt Original Estimate: 8h Remaining Estimate: 8h Some switches happen in the set of memtables and sstables of a cfstore: * when we switch the memtable, the current memtable is replaced by a new one and add to the memtablesPendingFlush * when a memtable is fully flushed, it is removed from memtablesPendingFlush and the newly created sstable is added to the set of active sstables. * after compaction, compacted memtables are removed from the active sstables and the compacted sstable is added instead Only the last of these operations is atomic. This ticket proposes to makes all of them atomic, using the idea of the View idea hinted by Stu in the comments of CASSANDRA-1954. The main reason for this is to fix CASSANDRA-2105. But I think that another benefit of this is to make reasoning about those operations easier. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2283) Streaming Old Format Data Fails in 0.7.3 after upgrade from 0.6.8
[ https://issues.apache.org/jira/browse/CASSANDRA-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012077#comment-13012077 ] Jonathan Ellis commented on CASSANDRA-2283: --- You're right, making it one-pass isn't feasible without writing the streamed row out to a temporary file first, since we don't have a way to rebuild the row-level bloom filter + block index. In other words, not really any more one-pass than the existing approach. Streaming Old Format Data Fails in 0.7.3 after upgrade from 0.6.8 - Key: CASSANDRA-2283 URL: https://issues.apache.org/jira/browse/CASSANDRA-2283 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.3 Environment: 0.7.3 upgraded from 0.6.8, Linux Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03, mixed mode) Reporter: Erik Onnen After successfully upgrading a 0.6.8 ring to 0.7.3, we needed to bootstrap in a new node relatively quickly. When starting the new node with an assigned token in auto bootstrap mode, we see the following exceptions on the new node: INFO [main] 2011-03-07 10:37:32,671 StorageService.java (line 505) Joining: sleeping 3 ms for pending range setup INFO [main] 2011-03-07 10:38:02,679 StorageService.java (line 505) Bootstrapping INFO [HintedHandoff:1] 2011-03-07 10:38:02,899 HintedHandOffManager.java (line 304) Started hinted handoff for endpoint /10.211.14.200 INFO [HintedHandoff:1] 2011-03-07 10:38:02,900 HintedHandOffManager.java (line 360) Finished hinted handoff of 0 rows to endpoint /10.211.14.200 INFO [CompactionExecutor:1] 2011-03-07 10:38:04,924 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuff-f-1 INFO [CompactionExecutor:1] 2011-03-07 10:38:05,390 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuff-f-2 INFO [CompactionExecutor:1] 2011-03-07 10:38:05,768 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid-f-1 INFO [CompactionExecutor:1] 2011-03-07 10:38:06,389 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid-f-2 INFO [CompactionExecutor:1] 2011-03-07 10:38:06,581 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid-f-3 ERROR [CompactionExecutor:1] 2011-03-07 10:38:07,056 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.io.EOFException at org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:65) at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:303) at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:923) at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:916) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) INFO [CompactionExecutor:1] 2011-03-07 10:38:08,480 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid-f-5 INFO [CompactionExecutor:1] 2011-03-07 10:38:08,582 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid_reg_idx-f-1 ERROR [CompactionExecutor:1] 2011-03-07 10:38:08,635 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.io.EOFException at org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:65) at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:303) at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:923) at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:916) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) ERROR [CompactionExecutor:1] 2011-03-07 10:38:08,666 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.io.EOFException at
[jira] [Commented] (CASSANDRA-2105) Fix the read race condition in CFStore for counters
[ https://issues.apache.org/jira/browse/CASSANDRA-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012081#comment-13012081 ] Hudson commented on CASSANDRA-2105: --- Integrated in Cassandra #810 (See [https://hudson.apache.org/hudson/job/Cassandra/810/]) Atomically switch cfstore memtables and sstables patch by slebresne; reviewed by jbellis for CASSANDRA-2284 (and CASSANDRA-2105) Fix the read race condition in CFStore for counters Key: CASSANDRA-2105 URL: https://issues.apache.org/jira/browse/CASSANDRA-2105 Project: Cassandra Issue Type: Bug Affects Versions: 0.8 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Labels: counters Fix For: 0.8 Attachments: 2115_option1_withLock.patch, 2115_option2_nolock.patch Original Estimate: 4h Remaining Estimate: 4h There is a (known) race condition during counter read. Indeed, for standard column family there is a small time during which a memtable is both active and pending flush and similarly a small time during which a 'memtable' is both pending flush and an active sstable. For counters that would imply sometime reconciling twice during a read the same counterColumn and thus over-counting. Current code changes this slightly by trading the possibility to count twice a given counterColumn by the possibility to miss a counterColumn. Thus it trades over-counts for under-counts. But this is no fix and there is no hope to offer clients any kind of guarantee on reads unless we fix this. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2283) Streaming Old Format Data Fails in 0.7.3 after upgrade from 0.6.8
[ https://issues.apache.org/jira/browse/CASSANDRA-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2283: -- Attachment: 2283.txt Patch to preserve version across streams. Also removes obsolete component field from PendingFile (we only stream Component.DATA). Streaming Old Format Data Fails in 0.7.3 after upgrade from 0.6.8 - Key: CASSANDRA-2283 URL: https://issues.apache.org/jira/browse/CASSANDRA-2283 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.1 Environment: 0.7.3 upgraded from 0.6.8, Linux Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03, mixed mode) Reporter: Erik Onnen Fix For: 0.7.5 Attachments: 2283.txt After successfully upgrading a 0.6.8 ring to 0.7.3, we needed to bootstrap in a new node relatively quickly. When starting the new node with an assigned token in auto bootstrap mode, we see the following exceptions on the new node: INFO [main] 2011-03-07 10:37:32,671 StorageService.java (line 505) Joining: sleeping 3 ms for pending range setup INFO [main] 2011-03-07 10:38:02,679 StorageService.java (line 505) Bootstrapping INFO [HintedHandoff:1] 2011-03-07 10:38:02,899 HintedHandOffManager.java (line 304) Started hinted handoff for endpoint /10.211.14.200 INFO [HintedHandoff:1] 2011-03-07 10:38:02,900 HintedHandOffManager.java (line 360) Finished hinted handoff of 0 rows to endpoint /10.211.14.200 INFO [CompactionExecutor:1] 2011-03-07 10:38:04,924 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuff-f-1 INFO [CompactionExecutor:1] 2011-03-07 10:38:05,390 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuff-f-2 INFO [CompactionExecutor:1] 2011-03-07 10:38:05,768 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid-f-1 INFO [CompactionExecutor:1] 2011-03-07 10:38:06,389 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid-f-2 INFO [CompactionExecutor:1] 2011-03-07 10:38:06,581 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid-f-3 ERROR [CompactionExecutor:1] 2011-03-07 10:38:07,056 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.io.EOFException at org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:65) at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:303) at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:923) at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:916) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) INFO [CompactionExecutor:1] 2011-03-07 10:38:08,480 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid-f-5 INFO [CompactionExecutor:1] 2011-03-07 10:38:08,582 SSTableReader.java (line 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid_reg_idx-f-1 ERROR [CompactionExecutor:1] 2011-03-07 10:38:08,635 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.io.EOFException at org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:65) at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:303) at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:923) at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:916) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) ERROR [CompactionExecutor:1] 2011-03-07 10:38:08,666 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.io.EOFException at org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:65) at
[jira] [Resolved] (CASSANDRA-2105) Fix the read race condition in CFStore for counters
[ https://issues.apache.org/jira/browse/CASSANDRA-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-2105. - Resolution: Fixed Fixed by CASSANDRA-2284 Fix the read race condition in CFStore for counters Key: CASSANDRA-2105 URL: https://issues.apache.org/jira/browse/CASSANDRA-2105 Project: Cassandra Issue Type: Bug Affects Versions: 0.8 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Labels: counters Fix For: 0.8 Attachments: 2115_option1_withLock.patch, 2115_option2_nolock.patch Original Estimate: 4h Remaining Estimate: 4h There is a (known) race condition during counter read. Indeed, for standard column family there is a small time during which a memtable is both active and pending flush and similarly a small time during which a 'memtable' is both pending flush and an active sstable. For counters that would imply sometime reconciling twice during a read the same counterColumn and thus over-counting. Current code changes this slightly by trading the possibility to count twice a given counterColumn by the possibility to miss a counterColumn. Thus it trades over-counts for under-counts. But this is no fix and there is no hope to offer clients any kind of guarantee on reads unless we fix this. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2284) Make changes to the set of memtables and sstables of a cfstore atomic
[ https://issues.apache.org/jira/browse/CASSANDRA-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-2284. - Resolution: Fixed Make changes to the set of memtables and sstables of a cfstore atomic -- Key: CASSANDRA-2284 URL: https://issues.apache.org/jira/browse/CASSANDRA-2284 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.8 Attachments: 0001-Make-memtable-and-sstable-switches-atomic-v2.patch, 0001-Make-memtable-and-sstable-switches-atomic-v3.patch, 0001-Make-memtable-and-sstable-switches-atomic.patch, 2284-v2-rebased.txt Original Estimate: 8h Remaining Estimate: 8h Some switches happen in the set of memtables and sstables of a cfstore: * when we switch the memtable, the current memtable is replaced by a new one and add to the memtablesPendingFlush * when a memtable is fully flushed, it is removed from memtablesPendingFlush and the newly created sstable is added to the set of active sstables. * after compaction, compacted memtables are removed from the active sstables and the compacted sstable is added instead Only the last of these operations is atomic. This ticket proposes to makes all of them atomic, using the idea of the View idea hinted by Stu in the comments of CASSANDRA-1954. The main reason for this is to fix CASSANDRA-2105. But I think that another benefit of this is to make reasoning about those operations easier. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2324) Repair transfers more data than necessary
[
https://issues.apache.org/jira/browse/CASSANDRA-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012087#comment-13012087
]
Jonathan Ellis commented on CASSANDRA-2324:
---
So what about this:
- change the atom of repair (in nodetool + StorageService) to be a single token
range, so it's unambiguous what we're repairing. This has the side benefit of
making it enormously easier to repair an entire cluster w/o doing redundant
work.
- provide backwards compatibility w/ existing repair command by splitting it
into RF repair ranges and waiting on each of those futures in StorageService
mbean
Repair transfers more data than necessary
-
Key: CASSANDRA-2324
URL: https://issues.apache.org/jira/browse/CASSANDRA-2324
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Sylvain Lebresne
Fix For: 0.7.5
To repro: 3 node cluster, stress.java 1M rows with -x KEYS and -l 2. The
index is enough to make some mutations drop (about 20-30k total in my tests).
Repair afterwards will repair a large amount of ranges the first time.
However, each subsequent run will repair the same set of small ranges every
time. INDEXED_RANGE_SLICE in stress never fully works. Counting rows with
sstablekeys shows there are 2M rows total as expected, however when trying to
count the indexed keys, I get exceptions like:
{noformat}
Exception in thread main java.io.IOException: Key out of order!
DecoratedKey(101571366040797913119296586470838356016,
0707ab782c5b5029d28a5e6d508ef72f0222528b5e28da3b7787492679dc51b96f868e0746073e54bc173be927049d0f51e25a6a95b3268213b8969abf40cea7d7)
DecoratedKey(12639574763031545147067490818595764132,
0bc414be3093348a2ad389ed28f18f0cc9a044b2e98587848a0d289dae13ed0ad479c74654900eeffc6236)
at
org.apache.cassandra.tools.SSTableExport.enumeratekeys(SSTableExport.java:206)
at
org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:388)
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2381) orphaned data files may be created during migration race
[ https://issues.apache.org/jira/browse/CASSANDRA-2381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012092#comment-13012092 ] Gary Dusbabek commented on CASSANDRA-2381: -- I'm seeing unit test failures in CliTest and DefsTest with v2 applied (never ran them on v1). orphaned data files may be created during migration race Key: CASSANDRA-2381 URL: https://issues.apache.org/jira/browse/CASSANDRA-2381 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.5 Attachments: 2381-0.8.txt, 2831-v2.txt We try to prevent creating orphans by locking Table.flusherLock in maybeSwitchMemtable and the Migration process, but since the actual writing is done asynchronously in Memtable.writeSortedContents there is a race window, where we acquire lock in maybeSwitch, we're not dropped so we queue the flush and release the lock, Migration does the drop, then Memtable writes itself out. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2373) Index Out Of Bounds during Validation Compaction (Repair)
[ https://issues.apache.org/jira/browse/CASSANDRA-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012095#comment-13012095 ] Sylvain Lebresne commented on CASSANDRA-2373: - Out of curiosity, are you able to major compact (or scrub) without problems ? I would be surprised if it was a validation specific thing, since it really doesn't do anything fancy as far as deserialization is involved. Index Out Of Bounds during Validation Compaction (Repair) - Key: CASSANDRA-2373 URL: https://issues.apache.org/jira/browse/CASSANDRA-2373 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.4 Environment: CentOS on EC2 Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Fix For: 0.7.5 Stack Trace is below: ERROR [CompactionExecutor:1] 2011-03-23 19:11:39,488 AbstractCassandraDaemon.java (line 112) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.lang.IndexOutOfBoundsException at java.nio.Buffer.checkIndex(Unknown Source) at java.nio.HeapByteBuffer.getInt(Unknown Source) at org.apache.cassandra.db.DeletedColumn.getLocalDeletionTime(DeletedColumn.java:57) at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:879) at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedColumnsOnly(ColumnFamilyStore.java:866) at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:857) at org.apache.cassandra.io.PrecompactedRow.init(PrecompactedRow.java:94) at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:147) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94) at org.apache.cassandra.db.CompactionManager.doValidationCompaction(CompactionManager.java:822) at org.apache.cassandra.db.CompactionManager.access$800(CompactionManager.java:56) at org.apache.cassandra.db.CompactionManager$6.call(CompactionManager.java:358) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2324) Repair transfers more data than necessary
[
https://issues.apache.org/jira/browse/CASSANDRA-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012097#comment-13012097
]
Sylvain Lebresne commented on CASSANDRA-2324:
-
Sounds good, will do.
Repair transfers more data than necessary
-
Key: CASSANDRA-2324
URL: https://issues.apache.org/jira/browse/CASSANDRA-2324
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Sylvain Lebresne
Fix For: 0.7.5
To repro: 3 node cluster, stress.java 1M rows with -x KEYS and -l 2. The
index is enough to make some mutations drop (about 20-30k total in my tests).
Repair afterwards will repair a large amount of ranges the first time.
However, each subsequent run will repair the same set of small ranges every
time. INDEXED_RANGE_SLICE in stress never fully works. Counting rows with
sstablekeys shows there are 2M rows total as expected, however when trying to
count the indexed keys, I get exceptions like:
{noformat}
Exception in thread main java.io.IOException: Key out of order!
DecoratedKey(101571366040797913119296586470838356016,
0707ab782c5b5029d28a5e6d508ef72f0222528b5e28da3b7787492679dc51b96f868e0746073e54bc173be927049d0f51e25a6a95b3268213b8969abf40cea7d7)
DecoratedKey(12639574763031545147067490818595764132,
0bc414be3093348a2ad389ed28f18f0cc9a044b2e98587848a0d289dae13ed0ad479c74654900eeffc6236)
at
org.apache.cassandra.tools.SSTableExport.enumeratekeys(SSTableExport.java:206)
at
org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:388)
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2370) unstable repo has disappeared from http://www.apache.org/dist/cassandra/debian/dists/
[ https://issues.apache.org/jira/browse/CASSANDRA-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna resolved CASSANDRA-2370. - Resolution: Fixed Asked in #asfinfra - sounds like something happened with the ownership of the dirs in there. It's fixed now. unstable repo has disappeared from http://www.apache.org/dist/cassandra/debian/dists/ - Key: CASSANDRA-2370 URL: https://issues.apache.org/jira/browse/CASSANDRA-2370 Project: Cassandra Issue Type: Bug Components: Packaging Environment: The entire internet Reporter: Brandon Williams Assignee: Eric Evans -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2373) Index Out Of Bounds during Validation Compaction (Repair)
[ https://issues.apache.org/jira/browse/CASSANDRA-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012101#comment-13012101 ] Benjamin Coverston commented on CASSANDRA-2373: --- I was able to scrub the node without any issues. Index Out Of Bounds during Validation Compaction (Repair) - Key: CASSANDRA-2373 URL: https://issues.apache.org/jira/browse/CASSANDRA-2373 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.4 Environment: CentOS on EC2 Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Fix For: 0.7.5 Stack Trace is below: ERROR [CompactionExecutor:1] 2011-03-23 19:11:39,488 AbstractCassandraDaemon.java (line 112) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.lang.IndexOutOfBoundsException at java.nio.Buffer.checkIndex(Unknown Source) at java.nio.HeapByteBuffer.getInt(Unknown Source) at org.apache.cassandra.db.DeletedColumn.getLocalDeletionTime(DeletedColumn.java:57) at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:879) at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedColumnsOnly(ColumnFamilyStore.java:866) at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:857) at org.apache.cassandra.io.PrecompactedRow.init(PrecompactedRow.java:94) at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:147) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94) at org.apache.cassandra.db.CompactionManager.doValidationCompaction(CompactionManager.java:822) at org.apache.cassandra.db.CompactionManager.access$800(CompactionManager.java:56) at org.apache.cassandra.db.CompactionManager$6.call(CompactionManager.java:358) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1902) Migrate cached pages during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012104#comment-13012104 ] T Jake Luciani commented on CASSANDRA-1902: --- I will test this to see how this handles hot reads before/during/after compaction Migrate cached pages during compaction --- Key: CASSANDRA-1902 URL: https://issues.apache.org/jira/browse/CASSANDRA-1902 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.1 Reporter: T Jake Luciani Assignee: T Jake Luciani Fix For: 0.7.5, 0.8 Attachments: 0001-CASSANDRA-1902-cache-migration-impl-with-config-option.txt, 1902-formatted.txt, 1902-per-column-migration-rebase2.txt, 1902-per-column-migration.txt, CASSANDRA-1902-v3.patch, CASSANDRA-1902-v4.patch, CASSANDRA-1902-v5.patch Original Estimate: 32h Time Spent: 56h Remaining Estimate: 0h Post CASSANDRA-1470 there is an opportunity to migrate cached pages from a pre-compacted CF during the compaction process. This is now important since CASSANDRA-1470 caches effectively nothing. For example an active CF being compacted hurts reads since nothing is cached in the new SSTable. The purpose of this ticket then is to make sure SOME data is cached from active CFs. This can be done my monitoring which Old SSTables are in the page cache and caching active rows in the New SStable. A simpler yet similar approach is described here: http://insights.oetiker.ch/linux/fadvise/ -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2373) Index Out Of Bounds during Validation Compaction (Repair)
[ https://issues.apache.org/jira/browse/CASSANDRA-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012106#comment-13012106 ] Sylvain Lebresne commented on CASSANDRA-2373: - Oh yeah, but actually now that I think about it, scrub is not a good test. What I suspect is that there is some crappy data, such that it is deserialized correctly as a tombstone, but the corruption is such that the value is 4 bytes. At least provided the stack trace, the problem is when accessing the deletion time, which scrub won't. I'd be more curious to see if a major compaction triggers it. If is is possible to get the sstable(s), I'd happily test it here. Index Out Of Bounds during Validation Compaction (Repair) - Key: CASSANDRA-2373 URL: https://issues.apache.org/jira/browse/CASSANDRA-2373 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.4 Environment: CentOS on EC2 Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Fix For: 0.7.5 Stack Trace is below: ERROR [CompactionExecutor:1] 2011-03-23 19:11:39,488 AbstractCassandraDaemon.java (line 112) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.lang.IndexOutOfBoundsException at java.nio.Buffer.checkIndex(Unknown Source) at java.nio.HeapByteBuffer.getInt(Unknown Source) at org.apache.cassandra.db.DeletedColumn.getLocalDeletionTime(DeletedColumn.java:57) at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:879) at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedColumnsOnly(ColumnFamilyStore.java:866) at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:857) at org.apache.cassandra.io.PrecompactedRow.init(PrecompactedRow.java:94) at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:147) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94) at org.apache.cassandra.db.CompactionManager.doValidationCompaction(CompactionManager.java:822) at org.apache.cassandra.db.CompactionManager.access$800(CompactionManager.java:56) at org.apache.cassandra.db.CompactionManager$6.call(CompactionManager.java:358) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2373) Index Out Of Bounds during Validation Compaction (Repair)
[ https://issues.apache.org/jira/browse/CASSANDRA-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012112#comment-13012112 ] Jonathan Ellis commented on CASSANDRA-2373: --- so we should enhance scrub to check fields? Index Out Of Bounds during Validation Compaction (Repair) - Key: CASSANDRA-2373 URL: https://issues.apache.org/jira/browse/CASSANDRA-2373 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.4 Environment: CentOS on EC2 Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Fix For: 0.7.5 Stack Trace is below: ERROR [CompactionExecutor:1] 2011-03-23 19:11:39,488 AbstractCassandraDaemon.java (line 112) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.lang.IndexOutOfBoundsException at java.nio.Buffer.checkIndex(Unknown Source) at java.nio.HeapByteBuffer.getInt(Unknown Source) at org.apache.cassandra.db.DeletedColumn.getLocalDeletionTime(DeletedColumn.java:57) at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:879) at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedColumnsOnly(ColumnFamilyStore.java:866) at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:857) at org.apache.cassandra.io.PrecompactedRow.init(PrecompactedRow.java:94) at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:147) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94) at org.apache.cassandra.db.CompactionManager.doValidationCompaction(CompactionManager.java:822) at org.apache.cassandra.db.CompactionManager.access$800(CompactionManager.java:56) at org.apache.cassandra.db.CompactionManager$6.call(CompactionManager.java:358) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2373) Index Out Of Bounds during Validation Compaction (Repair)
[ https://issues.apache.org/jira/browse/CASSANDRA-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012114#comment-13012114 ] Sylvain Lebresne commented on CASSANDRA-2373: - Yes, wouldn't be a bad idea. We could check that tombstone values are 4 bytes long (that almost could be an assert in ColumnSerializer.deserialize() actually, and I assume this is the problem here) and non negative value. Expiration time shouldn't be negative either. And if we validate the name and value through the validators, that would give an even better confidence that nothing is rotten. Index Out Of Bounds during Validation Compaction (Repair) - Key: CASSANDRA-2373 URL: https://issues.apache.org/jira/browse/CASSANDRA-2373 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.4 Environment: CentOS on EC2 Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Fix For: 0.7.5 Stack Trace is below: ERROR [CompactionExecutor:1] 2011-03-23 19:11:39,488 AbstractCassandraDaemon.java (line 112) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.lang.IndexOutOfBoundsException at java.nio.Buffer.checkIndex(Unknown Source) at java.nio.HeapByteBuffer.getInt(Unknown Source) at org.apache.cassandra.db.DeletedColumn.getLocalDeletionTime(DeletedColumn.java:57) at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:879) at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedColumnsOnly(ColumnFamilyStore.java:866) at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:857) at org.apache.cassandra.io.PrecompactedRow.init(PrecompactedRow.java:94) at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:147) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94) at org.apache.cassandra.db.CompactionManager.doValidationCompaction(CompactionManager.java:822) at org.apache.cassandra.db.CompactionManager.access$800(CompactionManager.java:56) at org.apache.cassandra.db.CompactionManager$6.call(CompactionManager.java:358) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2381) orphaned data files may be created during migration race
[ https://issues.apache.org/jira/browse/CASSANDRA-2381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2381: -- Attachment: 2381-v3.txt v3 grabs CF reference before removing it from CFMetadata map. tests pass. orphaned data files may be created during migration race Key: CASSANDRA-2381 URL: https://issues.apache.org/jira/browse/CASSANDRA-2381 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.5 Attachments: 2381-0.8.txt, 2381-v3.txt, 2831-v2.txt We try to prevent creating orphans by locking Table.flusherLock in maybeSwitchMemtable and the Migration process, but since the actual writing is done asynchronously in Memtable.writeSortedContents there is a race window, where we acquire lock in maybeSwitch, we're not dropped so we queue the flush and release the lock, Migration does the drop, then Memtable writes itself out. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2361) AES depends on java serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012119#comment-13012119 ] Sylvain Lebresne commented on CASSANDRA-2361: - This looks ok, but given the Merkle tree can be have a significant size (in term of nodes), it would maybe be worst it to use a byte to distinguish between a Inner and Leaf in the serialized form instead of the full class name. I understand that Java serialization wasn't necessarily very compact, but since we have a chance to do better. AES depends on java serialization - Key: CASSANDRA-2361 URL: https://issues.apache.org/jira/browse/CASSANDRA-2361 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.4 Reporter: Gary Dusbabek Assignee: Gary Dusbabek Priority: Minor Fix For: 0.7.5, 0.8 Attachments: 2361-part1.patch, v1-0001-ICompactSerializers-for-MerkleTrees.txt, v1-0002-git-rid-of-annoying-AES-emissions.txt 0.8 should be able to run in the same cluster as 0.7. AES uses java serialization which means that Token serialization stands a good chance of being brittle. This needs to be fixed. 1. place a hard-coded serialVersionUID for Token in 0.7.5. 2. have AES use ICompactSerializer in place of ObjectInputStream in 0.8. This would be a good opportunity to audit the code for imprudent uses of Object[Input|Output]Stream and get those cleaned up. Also, it will exercise the versioning code a bit. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2376) Both name an index iterators cast block offset to int
[ https://issues.apache.org/jira/browse/CASSANDRA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-2376. - Resolution: Fixed Committed as rev1086290. Both name an index iterators cast block offset to int - Key: CASSANDRA-2376 URL: https://issues.apache.org/jira/browse/CASSANDRA-2376 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.5 Attachments: 2376-v2.txt, 2376.txt This means that performing random access to the end of a large row will not work. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2396) Allow key name metadata
Allow key name metadata --- Key: CASSANDRA-2396 URL: https://issues.apache.org/jira/browse/CASSANDRA-2396 Project: Cassandra Issue Type: Improvement Components: API Reporter: Jonathan Ellis Assignee: Jon Hermes Fix For: 0.8 CASSANDRA-2311 allows typed keys. We also want to allow giving keys a name (that would need to conform to the CF comparator). This is required for Hive support; HIVE-1434 currently stores that metadata in an external repository but it would be cleaner to keep it in CFMetadata. This would allow CQL to support using the key name in select and where clauses, reducing the impedance mismatch between JDBC/CQL and JDBC/Hive. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2323) stress.java should not allow arbitrary arguments
[ https://issues.apache.org/jira/browse/CASSANDRA-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-2323: --- Attachment: CASSANDRA-2323.patch stress.java should not allow arbitrary arguments Key: CASSANDRA-2323 URL: https://issues.apache.org/jira/browse/CASSANDRA-2323 Project: Cassandra Issue Type: Bug Components: Contrib Affects Versions: 0.7.0 Reporter: Brandon Williams Assignee: Pavel Yaskevich Priority: Trivial Fix For: 0.7.5 Attachments: CASSANDRA-2323.patch This doesn't seem like a big deal, until you accidentally insert a space between a dash and it's flag, and it's at the point where the line wraps in your terminal. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2381) orphaned data files may be created during migration race
[ https://issues.apache.org/jira/browse/CASSANDRA-2381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012154#comment-13012154 ] Gary Dusbabek commented on CASSANDRA-2381: -- +1. orphaned data files may be created during migration race Key: CASSANDRA-2381 URL: https://issues.apache.org/jira/browse/CASSANDRA-2381 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.5 Attachments: 2381-0.8.txt, 2381-v3.txt, 2831-v2.txt We try to prevent creating orphans by locking Table.flusherLock in maybeSwitchMemtable and the Migration process, but since the actual writing is done asynchronously in Memtable.writeSortedContents there is a race window, where we acquire lock in maybeSwitch, we're not dropped so we queue the flush and release the lock, Migration does the drop, then Memtable writes itself out. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2261) During Compaction, Corrupt SSTables with rows that cause failures should be identified and blacklisted.
[ https://issues.apache.org/jira/browse/CASSANDRA-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Coverston updated CASSANDRA-2261: -- Attachment: (was: 2261.patch) During Compaction, Corrupt SSTables with rows that cause failures should be identified and blacklisted. --- Key: CASSANDRA-2261 URL: https://issues.apache.org/jira/browse/CASSANDRA-2261 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.6 Reporter: Benjamin Coverston Assignee: Benjamin Coverston Priority: Minor Labels: not_a_pony Fix For: 0.7.5 When a compaction of a set of SSTables fails because of corruption it will continue to try to compact that SSTable causing pending compactions to build up. One way to mitigate this problem would be to log the error, then identify the specific SSTable that caused the failure, subsequently blacklisting that SSTable and ensuring that it is no longer included in future compactions. For this we could simply store the problematic SSTable's name in memory. If it's not possible to identify the SSTable that caused the issue, then perhaps blacklisting the (ordered) permutation of SSTables to be compacted together is something that can be done to solve this problem in a more general case, and avoid issues where two (or more) SSTables have trouble compacting a particular row. For this option we would probably want to store the lists of the bad combinations in the system table somewhere s.t. these can survive a node failure (there have been a few cases where I have seen a compaction cause a node failure). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2261) During Compaction, Corrupt SSTables with rows that cause failures should be identified and blacklisted.
[ https://issues.apache.org/jira/browse/CASSANDRA-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Coverston updated CASSANDRA-2261: -- Attachment: 2261.patch Added a new patch that keeps the corrupted SSTable list resident and also added a new test. During Compaction, Corrupt SSTables with rows that cause failures should be identified and blacklisted. --- Key: CASSANDRA-2261 URL: https://issues.apache.org/jira/browse/CASSANDRA-2261 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.6 Reporter: Benjamin Coverston Assignee: Benjamin Coverston Priority: Minor Labels: not_a_pony Fix For: 0.7.5 Attachments: 2261.patch When a compaction of a set of SSTables fails because of corruption it will continue to try to compact that SSTable causing pending compactions to build up. One way to mitigate this problem would be to log the error, then identify the specific SSTable that caused the failure, subsequently blacklisting that SSTable and ensuring that it is no longer included in future compactions. For this we could simply store the problematic SSTable's name in memory. If it's not possible to identify the SSTable that caused the issue, then perhaps blacklisting the (ordered) permutation of SSTables to be compacted together is something that can be done to solve this problem in a more general case, and avoid issues where two (or more) SSTables have trouble compacting a particular row. For this option we would probably want to store the lists of the bad combinations in the system table somewhere s.t. these can survive a node failure (there have been a few cases where I have seen a compaction cause a node failure). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2361) AES depends on java serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek updated CASSANDRA-2361: - Attachment: v2-0002-get-rid-of-annoying-AES-emissions.txt v2-0001-ICompactSerializers-for-MerkleTrees.txt AES depends on java serialization - Key: CASSANDRA-2361 URL: https://issues.apache.org/jira/browse/CASSANDRA-2361 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.4 Reporter: Gary Dusbabek Assignee: Gary Dusbabek Priority: Minor Fix For: 0.7.5, 0.8 Attachments: 2361-part1.patch, v2-0001-ICompactSerializers-for-MerkleTrees.txt, v2-0002-get-rid-of-annoying-AES-emissions.txt 0.8 should be able to run in the same cluster as 0.7. AES uses java serialization which means that Token serialization stands a good chance of being brittle. This needs to be fixed. 1. place a hard-coded serialVersionUID for Token in 0.7.5. 2. have AES use ICompactSerializer in place of ObjectInputStream in 0.8. This would be a good opportunity to audit the code for imprudent uses of Object[Input|Output]Stream and get those cleaned up. Also, it will exercise the versioning code a bit. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2361) AES depends on java serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek updated CASSANDRA-2361: - Attachment: (was: v1-0001-ICompactSerializers-for-MerkleTrees.txt) AES depends on java serialization - Key: CASSANDRA-2361 URL: https://issues.apache.org/jira/browse/CASSANDRA-2361 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.4 Reporter: Gary Dusbabek Assignee: Gary Dusbabek Priority: Minor Fix For: 0.7.5, 0.8 Attachments: 2361-part1.patch, v2-0001-ICompactSerializers-for-MerkleTrees.txt, v2-0002-get-rid-of-annoying-AES-emissions.txt 0.8 should be able to run in the same cluster as 0.7. AES uses java serialization which means that Token serialization stands a good chance of being brittle. This needs to be fixed. 1. place a hard-coded serialVersionUID for Token in 0.7.5. 2. have AES use ICompactSerializer in place of ObjectInputStream in 0.8. This would be a good opportunity to audit the code for imprudent uses of Object[Input|Output]Stream and get those cleaned up. Also, it will exercise the versioning code a bit. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2361) AES depends on java serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012159#comment-13012159 ] Gary Dusbabek commented on CASSANDRA-2361: -- v2 implements Sylvain's suggestion for more compact serializations. AES depends on java serialization - Key: CASSANDRA-2361 URL: https://issues.apache.org/jira/browse/CASSANDRA-2361 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.4 Reporter: Gary Dusbabek Assignee: Gary Dusbabek Priority: Minor Fix For: 0.7.5, 0.8 Attachments: 2361-part1.patch, v2-0001-ICompactSerializers-for-MerkleTrees.txt, v2-0002-get-rid-of-annoying-AES-emissions.txt 0.8 should be able to run in the same cluster as 0.7. AES uses java serialization which means that Token serialization stands a good chance of being brittle. This needs to be fixed. 1. place a hard-coded serialVersionUID for Token in 0.7.5. 2. have AES use ICompactSerializer in place of ObjectInputStream in 0.8. This would be a good opportunity to audit the code for imprudent uses of Object[Input|Output]Stream and get those cleaned up. Also, it will exercise the versioning code a bit. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2376) Both name an index iterators cast block offset to int
[ https://issues.apache.org/jira/browse/CASSANDRA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012165#comment-13012165 ] Hudson commented on CASSANDRA-2376: --- Integrated in Cassandra-0.7 #410 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/410/]) Don't cast block offsets to int patch by jbellis; reviewed by slebresne for CASSANDRA-2376 Both name an index iterators cast block offset to int - Key: CASSANDRA-2376 URL: https://issues.apache.org/jira/browse/CASSANDRA-2376 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.5 Attachments: 2376-v2.txt, 2376.txt This means that performing random access to the end of a large row will not work. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2397) Improve or remove replicate-on-write setting
Improve or remove replicate-on-write setting Key: CASSANDRA-2397 URL: https://issues.apache.org/jira/browse/CASSANDRA-2397 Project: Cassandra Issue Type: Bug Reporter: Stu Hood The replicate on write setting breaks assumptions in various places in the codebase dealing with whether data will be replicated in a timely fashion. It's worthwhile to discuss whether we should go all-the-way on replicate-on-write, such that it is a fully supported feature, or whether we should remove it entirely. On one hand, ROW could be considered to be just another replication tunable like HH, RR and AES. On the other hand, a lazily replicating store is very rarely what you actually wanted. Open issues related to ROW are linked, but additionally, we'd need to: * Make the setting have an effect for standard column families * Change the default for ROW to enabled and properly warn of the effects -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1086343 - in /cassandra/branches/cassandra-0.7: ./ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/migration/
Author: jbellis Date: Mon Mar 28 19:05:32 2011 New Revision: 1086343 URL: http://svn.apache.org/viewvc?rev=1086343view=rev Log: fix migration race vs flush patch by jbellis; reviewed by gdusbabek for CASSANDRA-2381 Modified: cassandra/branches/cassandra-0.7/CHANGES.txt cassandra/branches/cassandra-0.7/NEWS.txt cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/ConsistencyLevel.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/RowIteratorFactory.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Table.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/migration/DropColumnFamily.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/migration/DropKeyspace.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/migration/Migration.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/migration/UpdateColumnFamily.java Modified: cassandra/branches/cassandra-0.7/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1086343r1=1086342r2=1086343view=diff == --- cassandra/branches/cassandra-0.7/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.7/CHANGES.txt Mon Mar 28 19:05:32 2011 @@ -21,6 +21,8 @@ * fsync statistics component on write (CASSANDRA-2382) * fix incorrect truncation of long to int when reading columns via block index (CASSANDRA-2376) + * fix race condition that could leave orphaned data files when + dropping CF or KS (CASSANDRA-2381) 0.7.4 Modified: cassandra/branches/cassandra-0.7/NEWS.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/NEWS.txt?rev=1086343r1=1086342r2=1086343view=diff == --- cassandra/branches/cassandra-0.7/NEWS.txt (original) +++ cassandra/branches/cassandra-0.7/NEWS.txt Mon Mar 28 19:05:32 2011 @@ -1,3 +1,18 @@ +0.7.5 += + +Upgrading +- +- Nothing specific to 0.7.4, but see 0.7.3 Upgrading if upgrading + from earlier than 0.7.1. + +Changes +--- +- system_update_column_family no longer snapshots before applying + the schema change. (_update_keyspace never did. _drop_keyspace + and _drop_column_family continue to snapshot.) + + 0.7.4 = Modified: cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/ConsistencyLevel.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/ConsistencyLevel.java?rev=1086343r1=1086342r2=1086343view=diff == --- cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/ConsistencyLevel.java (original) +++ cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/ConsistencyLevel.java Mon Mar 28 19:05:32 2011 @@ -32,13 +32,26 @@ import java.util.HashMap; import org.apache.thrift.TEnum; /** - * The ConsistencyLevel is an enum that controls both read and write behavior based on the ReplicationFactor of the keyspace. - * The different consistency levels have different meanings, depending on if you're doing a write or read - * operation. Note that if W + R ReplicationFactor, where W is the number of nodes to block for on write, and R - * the number to block for on reads, you will have strongly consistent behavior; that is, readers will always see the most - * recent write. Of these, the most interesting is to do QUORUM reads and writes, which gives you consistency while still - * allowing availability in the face of node failures up to half of ReplicationFactor. Of course if latency is more - * important than consistency then you can use lower values for either or both. + * The ConsistencyLevel is an enum that controls both read and write + * behavior based on the ReplicationFactor of the keyspace. The + * different consistency levels have different meanings, depending on + * if you're doing a write or read operation. + * + * If W + R ReplicationFactor, where W is the number of nodes to + * block for on write, and R the number to block for on reads, you + * will have strongly consistent behavior; that is, readers will + * always see the most recent write. Of these, the most interesting is + * to do QUORUM reads and writes, which gives you consistency while + * still allowing availability in the face of node failures up to half + * of ReplicationFactor. Of course if latency is more important than + * consistency then you can use lower values for either or both. + *
svn commit: r1086344 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/migration/
Author: jbellis Date: Mon Mar 28 19:07:03 2011 New Revision: 1086344 URL: http://svn.apache.org/viewvc?rev=1086344view=rev Log: merge from 0.7 Modified: cassandra/trunk/ (props changed) cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java cassandra/trunk/src/java/org/apache/cassandra/db/Memtable.java cassandra/trunk/src/java/org/apache/cassandra/db/Table.java cassandra/trunk/src/java/org/apache/cassandra/db/migration/DropColumnFamily.java cassandra/trunk/src/java/org/apache/cassandra/db/migration/DropKeyspace.java cassandra/trunk/src/java/org/apache/cassandra/db/migration/Migration.java cassandra/trunk/src/java/org/apache/cassandra/db/migration/UpdateColumnFamily.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 19:07:03 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7:1026516-1084291,1084660,1085372,1085538 +/cassandra/branches/cassandra-0.7:1026516-1084291,1084660,1085372,1085538,1086343 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 19:07:03 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1084291,1084660,1085372,1085538 +/cassandra/branches/cassandra-0.7/contrib:1026516-1084291,1084660,1085372,1085538,1086343 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 19:07:03 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1084291,1084660,1085372,1085538 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1084291,1084660,1085372,1085538,1086343 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 19:07:03 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1084291,1084660,1085372,1085538 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1084291,1084660,1085372,1085538,1086343 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/column_t.java:774578-792198 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
svn commit: r1086345 - in /cassandra/trunk: interface/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/avro/ src/java/org/apache/cassandra/config/ src/java/org/apache/cassandra/cql/ src/jav
Author: eevans
Date: Mon Mar 28 19:08:34 2011
New Revision: 1086345
URL: http://svn.apache.org/viewvc?rev=1086345view=rev
Log:
type validated row keys
Patch by Jon Hermes (w/ minor changes); reviewed by eevans for CASSANDRA-2311
Modified:
cassandra/trunk/interface/cassandra.thrift
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Constants.java
cassandra/trunk/src/avro/internode.genavro
cassandra/trunk/src/java/org/apache/cassandra/config/CFMetaData.java
cassandra/trunk/src/java/org/apache/cassandra/cql/QueryProcessor.java
cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java
cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java
Modified: cassandra/trunk/interface/cassandra.thrift
URL:
http://svn.apache.org/viewvc/cassandra/trunk/interface/cassandra.thrift?rev=1086345r1=1086344r2=1086345view=diff
==
--- cassandra/trunk/interface/cassandra.thrift (original)
+++ cassandra/trunk/interface/cassandra.thrift Mon Mar 28 19:08:34 2011
@@ -46,7 +46,7 @@ namespace rb CassandraThrift
# for every edit that doesn't result in a change to major/minor.
#
# See the Semantic Versioning Specification (SemVer) http://semver.org.
-const string VERSION = 19.4.0
+const string VERSION = 19.5.0
#
@@ -402,6 +402,7 @@ struct CfDef {
23: optional double memtable_operations_in_millions,
24: optional bool replicate_on_write=0,
25: optional double merge_shards_chance,
+26: optional string key_validation_class,
}
/* describes a keyspace. */
Modified:
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java?rev=1086345r1=1086344r2=1086345view=diff
==
---
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java
(original)
+++
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java
Mon Mar 28 19:08:34 2011
@@ -73,6 +73,7 @@ public class CfDef implements TBaseCfDe
private static final TField MEMTABLE_OPERATIONS_IN_MILLIONS_FIELD_DESC = new
TField(memtable_operations_in_millions, TType.DOUBLE, (short)23);
private static final TField REPLICATE_ON_WRITE_FIELD_DESC = new
TField(replicate_on_write, TType.BOOL, (short)24);
private static final TField MERGE_SHARDS_CHANCE_FIELD_DESC = new
TField(merge_shards_chance, TType.DOUBLE, (short)25);
+ private static final TField KEY_VALIDATION_CLASS_FIELD_DESC = new
TField(key_validation_class, TType.STRING, (short)26);
public String keyspace;
public String name;
@@ -96,6 +97,7 @@ public class CfDef implements TBaseCfDe
public double memtable_operations_in_millions;
public boolean replicate_on_write;
public double merge_shards_chance;
+ public String key_validation_class;
/** The set of fields this struct contains, along with convenience methods
for finding and manipulating them. */
public enum _Fields implements TFieldIdEnum {
@@ -120,7 +122,8 @@ public class CfDef implements TBaseCfDe
MEMTABLE_THROUGHPUT_IN_MB((short)22, memtable_throughput_in_mb),
MEMTABLE_OPERATIONS_IN_MILLIONS((short)23,
memtable_operations_in_millions),
REPLICATE_ON_WRITE((short)24, replicate_on_write),
-MERGE_SHARDS_CHANCE((short)25, merge_shards_chance);
+MERGE_SHARDS_CHANCE((short)25, merge_shards_chance),
+KEY_VALIDATION_CLASS((short)26, key_validation_class);
private static final MapString, _Fields byName = new HashMapString,
_Fields();
@@ -179,6 +182,8 @@ public class CfDef implements TBaseCfDe
return REPLICATE_ON_WRITE;
case 25: // MERGE_SHARDS_CHANCE
return MERGE_SHARDS_CHANCE;
+case 26: // KEY_VALIDATION_CLASS
+ return KEY_VALIDATION_CLASS;
default:
return null;
}
@@ -283,6 +288,8 @@ public class CfDef implements TBaseCfDe
new FieldValueMetaData(TType.BOOL)));
tmpMap.put(_Fields.MERGE_SHARDS_CHANCE, new
FieldMetaData(merge_shards_chance, TFieldRequirementType.OPTIONAL,
new FieldValueMetaData(TType.DOUBLE)));
+tmpMap.put(_Fields.KEY_VALIDATION_CLASS, new
FieldMetaData(key_validation_class, TFieldRequirementType.OPTIONAL,
+new FieldValueMetaData(TType.STRING)));
metaDataMap = Collections.unmodifiableMap(tmpMap);
FieldMetaData.addStructMetaDataMap(CfDef.class, metaDataMap);
}
@@ -359,6 +366,9 @@ public class CfDef implements TBaseCfDe
this.memtable_operations_in_millions =
other.memtable_operations_in_millions;
this.replicate_on_write = other.replicate_on_write;
this.merge_shards_chance = other.merge_shards_chance;
+if
[jira] [Commented] (CASSANDRA-2311) type validated row keys
[
https://issues.apache.org/jira/browse/CASSANDRA-2311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012179#comment-13012179
]
Eric Evans commented on CASSANDRA-2311:
---
This looks OK to me, but I found the name {{validateKeyInColumnFamily}}
confusing and so changed it to {{validateKeyType}} in the hopes it would make
things clearer (I won't swear that it does :)).
I also removed the logging statement from the try/catch in the validation
method, (I assume that was there for debugging purposes).
If you see a problem with either of these two edits, reopen the ticket and/or
ping me on IRC.
Thanks Jon!
type validated row keys
---
Key: CASSANDRA-2311
URL: https://issues.apache.org/jira/browse/CASSANDRA-2311
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Eric Evans
Assignee: Jon Hermes
Labels: cql
Fix For: 0.8
Attachments: 2311.txt
The idea here is to allow the assignment of a column-family-wide key type
used to perform validation, (similar to how default_validation_class does for
column values).
This should be as straightforward as extending the column family schema to
include the new attribute, and updating {{ThriftValidation.validateKey}} to
validate the key ({{AbstractType.validate}}).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1086350 - in /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db: ColumnFamilyStore.java Memtable.java
Author: jbellis
Date: Mon Mar 28 19:15:18 2011
New Revision: 1086350
URL: http://svn.apache.org/viewvc?rev=1086350view=rev
Log:
add actual dropped check to Memtable.flush for CASSANDRA-2381
patch by jbellis
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1086350r1=1086349r2=1086350view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Mon Mar 28 19:15:18 2011
@@ -709,10 +709,7 @@ public class ColumnFamilyStore implement
return null;
}
-boolean isDropped = isIndex()
- ? DatabaseDescriptor.getCFMetaData(table.name,
getParentColumnfamily()) == null
- :
DatabaseDescriptor.getCFMetaData(metadata.cfId) == null;
-if (isDropped)
+if (isDropped())
{
logger.debug(column family was dropped; no point in
flushing);
return null;
@@ -768,6 +765,13 @@ public class ColumnFamilyStore implement
}
}
+public boolean isDropped()
+{
+return isIndex()
+ ? DatabaseDescriptor.getCFMetaData(table.name,
getParentColumnfamily()) == null
+ : DatabaseDescriptor.getCFMetaData(metadata.cfId) == null;
+}
+
void switchBinaryMemtable(DecoratedKey key, ByteBuffer buffer)
{
binaryMemtable.set(new BinaryMemtable(this));
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java?rev=1086350r1=1086349r2=1086350view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
Mon Mar 28 19:15:18 2011
@@ -175,8 +175,11 @@ public class Memtable implements Compara
cfs.flushLock.lock();
try
{
-cfs.addSSTable(writeSortedContents());
-cfs.getMemtablesPendingFlush().remove(Memtable.this);
+if (!cfs.isDropped())
+{
+cfs.addSSTable(writeSortedContents());
+cfs.getMemtablesPendingFlush().remove(Memtable.this);
+}
}
finally
{
[jira] [Commented] (CASSANDRA-2394) Faulty hd kills cluster performance
[ https://issues.apache.org/jira/browse/CASSANDRA-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012181#comment-13012181 ] Brandon Williams commented on CASSANDRA-2394: - The dynamic snitch would fix this in a small cluster. In a larger cluster, where many nodes are being used as coordinators, each one will have to observe the faulty node performing badly, and this could be observed as a global degradation in performance, at least until the snitch on all coordinators decided to quit using the node. Faulty hd kills cluster performance --- Key: CASSANDRA-2394 URL: https://issues.apache.org/jira/browse/CASSANDRA-2394 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.4 Reporter: Thibaut Priority: Minor Fix For: 0.7.5 Hi, About every week, a node from our main cluster (100 nodes) has a faulty hd (Listing the cassandra data storage directoy triggers an input/output error). Whenever this occurs, I see many timeoutexceptions in our application on various nodes which cause everything to run very very slowly. Keyrange scans just timeout and will sometimes never succeed. If I stop cassandra on the faulty node, everything runs normal again. It would be great to have some kind of monitoring thread in cassandra which marks a node as down if there are multiple read/write errors to the data directories. A single faulty hd on 1 node shouldn't affect global cluster performance. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1086351 - /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Author: jbellis
Date: Mon Mar 28 19:17:19 2011
New Revision: 1086351
URL: http://svn.apache.org/viewvc?rev=1086351view=rev
Log:
r/m unnecessary isDropped check from maybeSwitchMemtable (the one in
Memtable.writeSortedContents it the important one)
patch by jbellis for CASSANDRA-2381
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1086351r1=1086350r2=1086351view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Mon Mar 28 19:17:19 2011
@@ -709,12 +709,6 @@ public class ColumnFamilyStore implement
return null;
}
-if (isDropped())
-{
-logger.debug(column family was dropped; no point in
flushing);
-return null;
-}
-
assert memtable == oldMemtable;
memtable.freeze();
final CommitLogSegment.CommitLogContext ctx = writeCommitLog ?
CommitLog.instance.getContext() : null;
[jira] [Commented] (CASSANDRA-2394) Faulty hd kills cluster performance
[ https://issues.apache.org/jira/browse/CASSANDRA-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012187#comment-13012187 ] Jonathan Ellis commented on CASSANDRA-2394: --- bq. until the snitch on all coordinators decided to quit using the node but shouldn't that be negligibly slower than in a small cluster, assuming there is enough query volume that each coordinator is routing some queries for the data in question? Faulty hd kills cluster performance --- Key: CASSANDRA-2394 URL: https://issues.apache.org/jira/browse/CASSANDRA-2394 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.4 Reporter: Thibaut Priority: Minor Fix For: 0.7.5 Hi, About every week, a node from our main cluster (100 nodes) has a faulty hd (Listing the cassandra data storage directoy triggers an input/output error). Whenever this occurs, I see many timeoutexceptions in our application on various nodes which cause everything to run very very slowly. Keyrange scans just timeout and will sometimes never succeed. If I stop cassandra on the faulty node, everything runs normal again. It would be great to have some kind of monitoring thread in cassandra which marks a node as down if there are multiple read/write errors to the data directories. A single faulty hd on 1 node shouldn't affect global cluster performance. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2398) Type specific compression
Type specific compression
-
Key: CASSANDRA-2398
URL: https://issues.apache.org/jira/browse/CASSANDRA-2398
Project: Cassandra
Issue Type: New Feature
Components: Core
Affects Versions: 0.8
Reporter: Stu Hood
Cassandra has a lot of locations that are ripe for type specific compression. A
short list:
Indexes
* Keys compressed as BytesType, which could default to LZO/LZMA
* Offsets (delta and varint encoding)
* Column names added by 2319
Data
* Keys, columns, timestamps: see
http://wiki.apache.org/cassandra/FileFormatDesignDoc
A basic interface for type specific compression could be as simple as:
{code:java}
public void compress(int version, IteratorByteBuffer from, int count,
DataOutput to) throws IOException
public void decompress(int version, DataInput from, ListByteBuffer to) throws
IOException
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2398) Type specific compression
[
https://issues.apache.org/jira/browse/CASSANDRA-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stu Hood updated CASSANDRA-2398:
Description:
Cassandra has a lot of locations that are ripe for type specific compression. A
short list:
Indexes
* Keys compressed as BytesType, which could default to LZO/LZMA
* Offsets (delta and varint encoding)
* Column names added by 2319
Data
* Keys, columns, timestamps: see
http://wiki.apache.org/cassandra/FileFormatDesignDoc
A basic interface for type specific compression could be as simple as:
{code:java}
public void compress(int version, IteratorByteBuffer from, int count,
DataOutput to) throws IOException
public void decompress(int version, DataInput from, ListByteBuffer to) throws
IOException
{code}
was:
Cassandra has a lot of locations that are ripe for type specific compression. A
short list:
Indexes
* Keys compressed as BytesType, which could default to LZO/LZMA
* Offsets (delta and varint encoding)
* Column names added by 2319
Data
* Keys, columns, timestamps: see
http://wiki.apache.org/cassandra/FileFormatDesignDoc
A basic interface for type specific compression could be as simple as:
{code:java}
public void compress(int version, IteratorByteBuffer from, int count,
DataOutput to) throws IOException
public void decompress(int version, DataInput from, ListByteBuffer to) throws
IOException
{code}
Type specific compression
-
Key: CASSANDRA-2398
URL: https://issues.apache.org/jira/browse/CASSANDRA-2398
Project: Cassandra
Issue Type: New Feature
Components: Core
Affects Versions: 0.8
Reporter: Stu Hood
Labels: compression
Cassandra has a lot of locations that are ripe for type specific compression.
A short list:
Indexes
* Keys compressed as BytesType, which could default to LZO/LZMA
* Offsets (delta and varint encoding)
* Column names added by 2319
Data
* Keys, columns, timestamps: see
http://wiki.apache.org/cassandra/FileFormatDesignDoc
A basic interface for type specific compression could be as simple as:
{code:java}
public void compress(int version, IteratorByteBuffer from, int count,
DataOutput to) throws IOException
public void decompress(int version, DataInput from, ListByteBuffer to)
throws IOException
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2398) Type specific compression
[
https://issues.apache.org/jira/browse/CASSANDRA-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stu Hood updated CASSANDRA-2398:
Attachment: example.diff
Attaching an example of noop compression.
The general idea is that AbstractType would implement LZO/LZMA by default, and
subclasses could override in the future to add actual type specificity. The
versioning in this example is MessagingService.version_, although we should
discuss whether that is the right one.
Type specific compression
-
Key: CASSANDRA-2398
URL: https://issues.apache.org/jira/browse/CASSANDRA-2398
Project: Cassandra
Issue Type: New Feature
Components: Core
Affects Versions: 0.8
Reporter: Stu Hood
Labels: compression
Attachments: example.diff
Cassandra has a lot of locations that are ripe for type specific compression.
A short list:
Indexes
* Keys compressed as BytesType, which could default to LZO/LZMA
* Offsets (delta and varint encoding)
* Column names added by 2319
Data
* Keys, columns, timestamps: see
http://wiki.apache.org/cassandra/FileFormatDesignDoc
A basic interface for type specific compression could be as simple as:
{code:java}
public void compress(int version, IteratorByteBuffer from, int count,
DataOutput to) throws IOException
public void decompress(int version, DataInput from, ListByteBuffer to)
throws IOException
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2381) orphaned data files may be created during migration race
[ https://issues.apache.org/jira/browse/CASSANDRA-2381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012201#comment-13012201 ] Hudson commented on CASSANDRA-2381: --- Integrated in Cassandra-0.7 #411 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/411/]) r/m unnecessary isDropped check from maybeSwitchMemtable (the one in Memtable.writeSortedContents it the important one) patch by jbellis for CASSANDRA-2381 add actual dropped check to Memtable.flush for CASSANDRA-2381 patch by jbellis fix migration race vs flush patch by jbellis; reviewed by gdusbabek for CASSANDRA-2381 orphaned data files may be created during migration race Key: CASSANDRA-2381 URL: https://issues.apache.org/jira/browse/CASSANDRA-2381 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.5 Attachments: 2381-0.8.txt, 2381-v3.txt, 2831-v2.txt We try to prevent creating orphans by locking Table.flusherLock in maybeSwitchMemtable and the Migration process, but since the actual writing is done asynchronously in Memtable.writeSortedContents there is a race window, where we acquire lock in maybeSwitch, we're not dropped so we queue the flush and release the lock, Migration does the drop, then Memtable writes itself out. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2367) Cleanup conversions between bytes and strings
[ https://issues.apache.org/jira/browse/CASSANDRA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012213#comment-13012213 ] Hudson commented on CASSANDRA-2367: --- Integrated in Cassandra #812 (See [https://hudson.apache.org/hudson/job/Cassandra/812/]) Cleanup conversions between bytes and strings - Key: CASSANDRA-2367 URL: https://issues.apache.org/jira/browse/CASSANDRA-2367 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.7.5 Attachments: 0001-Cleanup-bytes-string-conversions.patch Original Estimate: 2h Remaining Estimate: 2h There is a bit of inconsistency in our conversions between ByteBuffers and Strings. For instance, ByteBufferUtil.string() uses as a default the java default charset, while ByteBufferUtil.bytes(String) assumes UTF8. Moreover, a number of places in the code don't use those functions and uses getBytes() directly. There again, we often encode with the default charset but decode in UTF8 or the contrary. Using the default charset is probably a bad idea anyway, since this depends on the actual system the node is running on and could lead to a stupid bug when running in heterogeneous systems. This ticket proposes to always assume UTF8 all over the place (and tries to use the ByteBufferUtil as much as possible to help with that). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2311) type validated row keys
[
https://issues.apache.org/jira/browse/CASSANDRA-2311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012214#comment-13012214
]
Hudson commented on CASSANDRA-2311:
---
Integrated in Cassandra #812 (See
[https://hudson.apache.org/hudson/job/Cassandra/812/])
type validated row keys
Patch by Jon Hermes (w/ minor changes); reviewed by eevans for CASSANDRA-2311
type validated row keys
---
Key: CASSANDRA-2311
URL: https://issues.apache.org/jira/browse/CASSANDRA-2311
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Eric Evans
Assignee: Jon Hermes
Labels: cql
Fix For: 0.8
Attachments: 2311.txt
The idea here is to allow the assignment of a column-family-wide key type
used to perform validation, (similar to how default_validation_class does for
column values).
This should be as straightforward as extending the column family schema to
include the new attribute, and updating {{ThriftValidation.validateKey}} to
validate the key ({{AbstractType.validate}}).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1086400 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/
Author: jbellis Date: Mon Mar 28 21:00:20 2011 New Revision: 1086400 URL: http://svn.apache.org/viewvc?rev=1086400view=rev Log: merge from 0.7 Modified: cassandra/trunk/ (props changed) cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/db/Memtable.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:00:20 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7:1026516-1084291,1084660,1085372,1085538,1086343 +/cassandra/branches/cassandra-0.7:1026516-1084291,1084660,1085372,1085538,1086343,1086350-1086351 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:00:20 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1084291,1084660,1085372,1085538,1086343 +/cassandra/branches/cassandra-0.7/contrib:1026516-1084291,1084660,1085372,1085538,1086343,1086350-1086351 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:00:20 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1084291,1084660,1085372,1085538,1086343 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1084291,1084660,1085372,1085538,1086343,1086350-1086351 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:00:20 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1084291,1084660,1085372,1085538,1086343 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1084291,1084660,1085372,1085538,1086343,1086350-1086351 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/column_t.java:774578-792198 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:00:20 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000
svn commit: r1086403 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/utils/ test/system/
Author: jbellis Date: Mon Mar 28 21:09:54 2011 New Revision: 1086403 URL: http://svn.apache.org/viewvc?rev=1086403view=rev Log: merge from 0.7 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/utils/ByteBufferUtil.java cassandra/trunk/test/system/test_thrift_server.py Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:09:54 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7:1026516-1084291,1084660,1085372,1085538,1086343,1086350-1086351 +/cassandra/branches/cassandra-0.7:1026516-1084660,1085372,1085538,1086343,1086350-1086351 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1086403r1=1086402r2=1086403view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Mon Mar 28 21:09:54 2011 @@ -32,6 +32,7 @@ * fsync statistics component on write (CASSANDRA-2382) * fix encoding bugs in HintedHandoffManager, SystemTable when default charset is not UTF8 (CASSANDRA-2367) + * fix potential infinite loop in ByteBufferUtil.inputStream (CASSANDRA-2365) 0.7.4 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:09:54 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1084291,1084660,1085372,1085538,1086343,1086350-1086351 +/cassandra/branches/cassandra-0.7/contrib:1026516-1084660,1085372,1085538,1086343,1086350-1086351 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:09:54 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1084291,1084660,1085372,1085538,1086343,1086350-1086351 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1084660,1085372,1085538,1086343,1086350-1086351 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:09:54 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1084291,1084660,1085372,1085538,1086343,1086350-1086351 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1084660,1085372,1085538,1086343,1086350-1086351 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654
svn commit: r1086404 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/cli/ src/java/org/apache/cassandra/dht/ src/java/org/apache
Author: jbellis Date: Mon Mar 28 21:11:19 2011 New Revision: 1086404 URL: http://svn.apache.org/viewvc?rev=1086404view=rev Log: merge from 0.7 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java cassandra/trunk/src/java/org/apache/cassandra/dht/BigIntegerToken.java cassandra/trunk/src/java/org/apache/cassandra/dht/BytesToken.java cassandra/trunk/src/java/org/apache/cassandra/dht/LocalToken.java cassandra/trunk/src/java/org/apache/cassandra/dht/StringToken.java cassandra/trunk/src/java/org/apache/cassandra/service/AntiEntropyService.java cassandra/trunk/src/java/org/apache/cassandra/streaming/StreamReplyVerbHandler.java cassandra/trunk/test/unit/org/apache/cassandra/cli/CliTest.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:11:19 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7:1026516-1084660,1085372,1085538,1086343,1086350-1086351 +/cassandra/branches/cassandra-0.7:1026516-1085372,1085538,1086343,1086350-1086351 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1086404r1=1086403r2=1086404view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Mon Mar 28 21:11:19 2011 @@ -33,6 +33,7 @@ * fix encoding bugs in HintedHandoffManager, SystemTable when default charset is not UTF8 (CASSANDRA-2367) * fix potential infinite loop in ByteBufferUtil.inputStream (CASSANDRA-2365) + * allow negative numbers in the cli (CASSANDRA-2358) 0.7.4 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:11:19 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1084660,1085372,1085538,1086343,1086350-1086351 +/cassandra/branches/cassandra-0.7/contrib:1026516-1085372,1085538,1086343,1086350-1086351 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:11:19 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1084660,1085372,1085538,1086343,1086350-1086351 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1085372,1085538,1086343,1086350-1086351 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:11:19 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure of cassandra-trunk on ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1187 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: Build Source Stamp: [branch cassandra/trunk] 1086404 Blamelist: jbellis BUILD FAILED: failed compile sincerely, -The Buildbot
svn commit: r1086406 - in /cassandra/trunk: ./ conf/ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/cli/ src/java/org/apache/cassandra/db/columniterator/
Author: jbellis
Date: Mon Mar 28 21:21:29 2011
New Revision: 1086406
URL: http://svn.apache.org/viewvc?rev=1086406view=rev
Log:
merge from 0.7
Modified:
cassandra/trunk/ (props changed)
cassandra/trunk/CHANGES.txt
cassandra/trunk/conf/schema-sample.txt
cassandra/trunk/contrib/ (props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
(props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
(props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
(props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
(props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
(props changed)
cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g
cassandra/trunk/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
cassandra/trunk/src/java/org/apache/cassandra/db/columniterator/SSTableNamesIterator.java
cassandra/trunk/src/java/org/apache/cassandra/hadoop/streaming/AvroOutputReader.java
cassandra/trunk/src/java/org/apache/cassandra/io/sstable/IndexHelper.java
cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTable.java
cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java
cassandra/trunk/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Mar 28 21:21:29 2011
@@ -1,5 +1,5 @@
/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000
-/cassandra/branches/cassandra-0.7:1026516-1085372,1085538,1086343,1086350-1086351
+/cassandra/branches/cassandra-0.7:1026516-1086404
/cassandra/branches/cassandra-0.7.0:1053690-1055654
/cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689
/incubator/cassandra/branches/cassandra-0.3:774578-796573
Modified: cassandra/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1086406r1=1086405r2=1086406view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Mon Mar 28 21:21:29 2011
@@ -34,6 +34,8 @@
charset is not UTF8 (CASSANDRA-2367)
* fix potential infinite loop in ByteBufferUtil.inputStream (CASSANDRA-2365)
* allow negative numbers in the cli (CASSANDRA-2358)
+ * fix incorrect truncation of long to int when reading columns via block
+ index (CASSANDRA-2376)
0.7.4
Modified: cassandra/trunk/conf/schema-sample.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/conf/schema-sample.txt?rev=1086406r1=1086405r2=1086406view=diff
==
--- cassandra/trunk/conf/schema-sample.txt (original)
+++ cassandra/trunk/conf/schema-sample.txt Mon Mar 28 21:21:29 2011
@@ -53,7 +53,8 @@ create column family Super3
and comment = 'A column family with supercolumns, whose column names are
Longs (8 bytes)';
create column family Indexed1
-with default_validation_class = LongType
+with comparator = UTF8Type
+and default_validation_class = LongType
and column_metadata = [{
column_name : birthdate,
validation_class : LongType,
Propchange: cassandra/trunk/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Mar 28 21:21:29 2011
@@ -1,5 +1,5 @@
/cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009
-/cassandra/branches/cassandra-0.7/contrib:1026516-1085372,1085538,1086343,1086350-1086351
+/cassandra/branches/cassandra-0.7/contrib:1026516-1086404
/cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654
/cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689
/incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573
Propchange:
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Mar 28 21:21:29 2011
@@ -1,5 +1,5 @@
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1085372,1085538,1086343,1086350-1086351
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1086404
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654
[jira] [Reopened] (CASSANDRA-2358) CLI doesn't handle inserting negative integers
[
https://issues.apache.org/jira/browse/CASSANDRA-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reopened CASSANDRA-2358:
---
merged to trunk but CliTest fails. Can you fix?
CLI doesn't handle inserting negative integers
--
Key: CASSANDRA-2358
URL: https://issues.apache.org/jira/browse/CASSANDRA-2358
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 0.7.0
Reporter: Tyler Hobbs
Assignee: Pavel Yaskevich
Priority: Trivial
Fix For: 0.7.5
Attachments: CASSANDRA-2358.patch
Original Estimate: 0.5h
Remaining Estimate: 0.5h
The CLI raises a syntax error when trying to insert negative integers:
{noformat}
[default@Keyspace1] set StandardInteger['key'][-12] = 'val';
Syntax error at position 28: mismatched character '1' expecting '-'
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2358) CLI doesn't handle inserting negative integers
[
https://issues.apache.org/jira/browse/CASSANDRA-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012233#comment-13012233
]
Pavel Yaskevich commented on CASSANDRA-2358:
Can you remove it from trunk for now? I will create a separate version of this
patch for the trunk.
CLI doesn't handle inserting negative integers
--
Key: CASSANDRA-2358
URL: https://issues.apache.org/jira/browse/CASSANDRA-2358
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 0.7.0
Reporter: Tyler Hobbs
Assignee: Pavel Yaskevich
Priority: Trivial
Fix For: 0.7.5
Attachments: CASSANDRA-2358.patch
Original Estimate: 0.5h
Remaining Estimate: 0.5h
The CLI raises a syntax error when trying to insert negative integers:
{noformat}
[default@Keyspace1] set StandardInteger['key'][-12] = 'val';
Syntax error at position 28: mismatched character '1' expecting '-'
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1086411 - /cassandra/branches/cassandra-0.7/CHANGES.txt
Author: jbellis Date: Mon Mar 28 21:33:08 2011 New Revision: 1086411 URL: http://svn.apache.org/viewvc?rev=1086411view=rev Log: clean up CHANGES Modified: cassandra/branches/cassandra-0.7/CHANGES.txt Modified: cassandra/branches/cassandra-0.7/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1086411r1=1086410r2=1086411view=diff == --- cassandra/branches/cassandra-0.7/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.7/CHANGES.txt Mon Mar 28 21:33:08 2011 @@ -289,11 +289,11 @@ * upgrade to SLF4J 1.6.1 * fix ByteBuffer bug in ExpiringColumn.updateDigest (CASSANDRA-1679) * fix IntegerType.getString (CASSANDRA-1681) - * log tpstats when dropping messages (CASSANDRA-1660) * make -Djava.net.preferIPv4Stack=true the default (CASSANDRA-628) * add INTERNAL_RESPONSE verb to differentiate from responses related to client requests (CASSANDRA-1685) * log tpstats when dropping messages (CASSANDRA-1660) + * include unreachable nodes in describeSchemaVersions (CASSANDRA-1678) * Avoid dropping messages off the client request path (CASSANDRA-1676) * fix jna errno reporting (CASSANDRA-1694) * add friendlier error for UnknownHostException on startup (CASSANDRA-1697)
[jira] [Created] (CASSANDRA-2399) Do not impact page cache during streaming
Do not impact page cache during streaming - Key: CASSANDRA-2399 URL: https://issues.apache.org/jira/browse/CASSANDRA-2399 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet Priority: Minor If we could do similar work we have done for compaction on streaming for new nodes, it would help to not blow out page cache. This might not be possible since we are using transferTo() but we should at least investigate. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1086416 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/
Author: jbellis Date: Mon Mar 28 21:55:32 2011 New Revision: 1086416 URL: http://svn.apache.org/viewvc?rev=1086416view=rev Log: clean up CHANGES Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:55:32 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7:1026516-1086404 +/cassandra/branches/cassandra-0.7:1026516-1086415 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1086416r1=1086415r2=1086416view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Mon Mar 28 21:55:32 2011 @@ -29,13 +29,15 @@ the same threshold for TTL expiration (CASSANDRA-2349) * fix race when iterating CFs during add/drop (CASSANDRA-2350) * add ConsistencyLevel command to CLI (CASSANDRA-2354) - * fsync statistics component on write (CASSANDRA-2382) + * fix potential infinite loop in ByteBufferUtil.inputStream (CASSANDRA-2365) * fix encoding bugs in HintedHandoffManager, SystemTable when default charset is not UTF8 (CASSANDRA-2367) - * fix potential infinite loop in ByteBufferUtil.inputStream (CASSANDRA-2365) * allow negative numbers in the cli (CASSANDRA-2358) + * fsync statistics component on write (CASSANDRA-2382) * fix incorrect truncation of long to int when reading columns via block index (CASSANDRA-2376) + * fix race condition that could leave orphaned data files when + dropping CF or KS (CASSANDRA-2381) 0.7.4 @@ -51,6 +53,8 @@ * add configurable row limit to Pig loadfunc (CASSANDRA-2276) * validate column values in batches as well as single-Column inserts (CASSANDRA-2259) + * move sample schema from cassandra.yaml to schema-sample.txt, + a cli scripts (CASSANDRA-2007) * avoid writing empty rows when scrubbing tombstoned rows (CASSANDRA-2296) * fix assertion error in range and index scans for CL ALL (CASSANDRA-2282) @@ -135,8 +139,8 @@ (CASSANDRA-1962) * allow specifying specific SSTables to compact from JMX (CASSANDRA-1963) * fix race condition in MessagingService.targets (CASSANDRA-1959, 2094, 2081) - * zero-copy reads (CASSANDRA-1714) * refuse to open sstables from a future version (CASSANDRA-1935) + * zero-copy reads (CASSANDRA-1714) * fix copy bounds for word Text in wordcount demo (CASSANDRA-1993) * fixes for contrib/javautils (CASSANDRA-1979) * check more frequently for memtable expiration (CASSANDRA-2000) @@ -300,7 +304,6 @@ * upgrade to SLF4J 1.6.1 * fix ByteBuffer bug in ExpiringColumn.updateDigest (CASSANDRA-1679) * fix IntegerType.getString (CASSANDRA-1681) - * log tpstats when dropping messages (CASSANDRA-1660) * make -Djava.net.preferIPv4Stack=true the default (CASSANDRA-628) * add INTERNAL_RESPONSE verb to differentiate from responses related to client requests (CASSANDRA-1685) Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:55:32 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1086404 +/cassandra/branches/cassandra-0.7/contrib:1026516-1086415 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Mar 28 21:55:32 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000
svn commit: r1086417 - in /cassandra/trunk: CHANGES.txt test/unit/org/apache/cassandra/cli/CliTest.java
Author: jbellis
Date: Mon Mar 28 21:58:59 2011
New Revision: 1086417
URL: http://svn.apache.org/viewvc?rev=1086417view=rev
Log:
revert merge of #2358
Modified:
cassandra/trunk/CHANGES.txt
cassandra/trunk/test/unit/org/apache/cassandra/cli/CliTest.java
Modified: cassandra/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1086417r1=1086416r2=1086417view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Mon Mar 28 21:58:59 2011
@@ -32,7 +32,6 @@
* fix potential infinite loop in ByteBufferUtil.inputStream (CASSANDRA-2365)
* fix encoding bugs in HintedHandoffManager, SystemTable when default
charset is not UTF8 (CASSANDRA-2367)
- * allow negative numbers in the cli (CASSANDRA-2358)
* fsync statistics component on write (CASSANDRA-2382)
* fix incorrect truncation of long to int when reading columns via block
index (CASSANDRA-2376)
Modified: cassandra/trunk/test/unit/org/apache/cassandra/cli/CliTest.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/cli/CliTest.java?rev=1086417r1=1086416r2=1086417view=diff
==
--- cassandra/trunk/test/unit/org/apache/cassandra/cli/CliTest.java (original)
+++ cassandra/trunk/test/unit/org/apache/cassandra/cli/CliTest.java Mon Mar 28
21:58:59 2011
@@ -40,11 +40,7 @@ public class CliTest extends CleanupHelp
create column family CF1 with comparator=UTF8Type and
column_metadata=[{ column_name:world, validation_class:IntegerType,
index_type:0, index_name:IdxName }, { column_name:world2,
validation_class:LongType, index_type:KEYS, index_name:LongIdxName}];,
assume CF1 keys as utf8;,
set CF1[hello][world] = 123848374878933948398384;,
-set CF1[hello][-31337] = 'some string value';,
-get CF1[hello][-31337];,
get CF1[hello][world];,
-set CF1[hello][-31337] = -23876;,
-set CF1[hello][-31337] = long(-23876);,
set CF1[hello][world2] = 15;,
get CF1 where world2 = long(15);,
get cF1 where world2 = long(15);,
@@ -88,10 +84,6 @@ public class CliTest extends CleanupHelp
del SCF1['hello'][1][];,
get SCF1['hello'][1][];,
set SCF1['hello'][1][] = Long(1234);,
-set SCF1['hello'][-1][-12] = Long(5678);,
-get SCF1['hello'][-1][-12];,
-set SCF1['hello'][-1][-12] = -340897;,
-set SCF1['hello'][-1][-12] = integer(-340897);,
del SCF1['hello'][];,
get SCF1['hello'][1][];,
create column family Counter1 with comparator=UTF8Type and
default_validation_class=CounterColumnType;,
[jira] [Commented] (CASSANDRA-2358) CLI doesn't handle inserting negative integers
[
https://issues.apache.org/jira/browse/CASSANDRA-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012249#comment-13012249
]
Jonathan Ellis commented on CASSANDRA-2358:
---
removed
CLI doesn't handle inserting negative integers
--
Key: CASSANDRA-2358
URL: https://issues.apache.org/jira/browse/CASSANDRA-2358
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 0.7.0
Reporter: Tyler Hobbs
Assignee: Pavel Yaskevich
Priority: Trivial
Fix For: 0.7.5
Attachments: CASSANDRA-2358.patch
Original Estimate: 0.5h
Remaining Estimate: 0.5h
The CLI raises a syntax error when trying to insert negative integers:
{noformat}
[default@Keyspace1] set StandardInteger['key'][-12] = 'val';
Syntax error at position 28: mismatched character '1' expecting '-'
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build of cassandra-trunk on ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1190 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: Build Source Stamp: [branch cassandra/trunk] 1086417 Blamelist: jbellis Build succeeded! sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-2398) Type specific compression
[
https://issues.apache.org/jira/browse/CASSANDRA-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012251#comment-13012251
]
Stu Hood commented on CASSANDRA-2398:
-
Actually, one restriction we should put on the output blob is that it should
start with a length for skipability.
Type specific compression
-
Key: CASSANDRA-2398
URL: https://issues.apache.org/jira/browse/CASSANDRA-2398
Project: Cassandra
Issue Type: New Feature
Components: Core
Affects Versions: 0.8
Reporter: Stu Hood
Labels: compression
Attachments: example.diff
Cassandra has a lot of locations that are ripe for type specific compression.
A short list:
Indexes
* Keys compressed as BytesType, which could default to LZO/LZMA
* Offsets (delta and varint encoding)
* Column names added by 2319
Data
* Keys, columns, timestamps: see
http://wiki.apache.org/cassandra/FileFormatDesignDoc
A basic interface for type specific compression could be as simple as:
{code:java}
public void compress(int version, IteratorByteBuffer from, int count,
DataOutput to) throws IOException
public void decompress(int version, DataInput from, ListByteBuffer to)
throws IOException
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2358) CLI doesn't handle inserting negative integers
[
https://issues.apache.org/jira/browse/CASSANDRA-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012250#comment-13012250
]
Pavel Yaskevich commented on CASSANDRA-2358:
Thanks!
CLI doesn't handle inserting negative integers
--
Key: CASSANDRA-2358
URL: https://issues.apache.org/jira/browse/CASSANDRA-2358
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 0.7.0
Reporter: Tyler Hobbs
Assignee: Pavel Yaskevich
Priority: Trivial
Fix For: 0.7.5
Attachments: CASSANDRA-2358.patch
Original Estimate: 0.5h
Remaining Estimate: 0.5h
The CLI raises a syntax error when trying to insert negative integers:
{noformat}
[default@Keyspace1] set StandardInteger['key'][-12] = 'val';
Syntax error at position 28: mismatched character '1' expecting '-'
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of ConfigurationNotes by JonHermes
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The ConfigurationNotes page has been changed by JonHermes.
http://wiki.apache.org/cassandra/ConfigurationNotes?action=diffrev1=7rev2=8
--
* update convertTo{Thrift/Avro}()
- * update apply()... this is the important method.
+ * update apply() (a.k.a. applyAvroMigrationChangesToCurrentCFMD)
+
+ * update convertToCFMetaData (a.k.a. convert thrift to CFMD and validate it)
Done. Then, whenever you need the option in code, use
DD.getCFMD(ksname,cfname).getFoo();
[Cassandra Wiki] Update of ConfigurationNotes by JonHermes
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The ConfigurationNotes page has been changed by JonHermes. http://wiki.apache.org/cassandra/ConfigurationNotes?action=diffrev1=8rev2=9 -- * update convertToCFMetaData (a.k.a. convert thrift to CFMD and validate it) + * if desired, add new option to CLI add/update CF + + * if desired, create a JMX call to CFS to change it at runtime + Done. Then, whenever you need the option in code, use DD.getCFMD(ksname,cfname).getFoo();
[jira] [Updated] (CASSANDRA-2396) Allow key name metadata
[ https://issues.apache.org/jira/browse/CASSANDRA-2396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Hermes updated CASSANDRA-2396: -- Attachment: 2396.txt Patch adds the prop. If CLI support and/or JMX calls are needed to expose this even further, they can be added on this ticket. It would be nice to link the CQL dep. to this ticket in case something comes up. Allow key name metadata --- Key: CASSANDRA-2396 URL: https://issues.apache.org/jira/browse/CASSANDRA-2396 Project: Cassandra Issue Type: Improvement Components: API Reporter: Jonathan Ellis Assignee: Jon Hermes Fix For: 0.8 Attachments: 2396.txt CASSANDRA-2311 allows typed keys. We also want to allow giving keys a name (that would need to conform to the CF comparator). This is required for Hive support; HIVE-1434 currently stores that metadata in an external repository but it would be cleaner to keep it in CFMetadata. This would allow CQL to support using the key name in select and where clauses, reducing the impedance mismatch between JDBC/CQL and JDBC/Hive. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2396) Allow key name metadata
[ https://issues.apache.org/jira/browse/CASSANDRA-2396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012266#comment-13012266 ] Jon Hermes commented on CASSANDRA-2396: --- Unrelated to this ticket but stumbled upon, moved convertToCFMD(thrift CfDef) into CFMD. It had no business being in CassandraServer. Allow key name metadata --- Key: CASSANDRA-2396 URL: https://issues.apache.org/jira/browse/CASSANDRA-2396 Project: Cassandra Issue Type: Improvement Components: API Reporter: Jonathan Ellis Assignee: Jon Hermes Fix For: 0.8 Attachments: 2396.txt CASSANDRA-2311 allows typed keys. We also want to allow giving keys a name (that would need to conform to the CF comparator). This is required for Hive support; HIVE-1434 currently stores that metadata in an external repository but it would be cleaner to keep it in CFMetadata. This would allow CQL to support using the key name in select and where clauses, reducing the impedance mismatch between JDBC/CQL and JDBC/Hive. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2400) Resolve Maven Ant Tasks from local Maven repository if present
Resolve Maven Ant Tasks from local Maven repository if present -- Key: CASSANDRA-2400 URL: https://issues.apache.org/jira/browse/CASSANDRA-2400 Project: Cassandra Issue Type: Improvement Reporter: Stephen Connolly Priority: Minor Attachments: MCASSANDRA-tweaks.patch To aid with testing using newer versions of Maven ANT Tasks it can be helpful to copy the version from the local repository if present rather than always downloading -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2400) Resolve Maven Ant Tasks from local Maven repository if present
[ https://issues.apache.org/jira/browse/CASSANDRA-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Connolly updated CASSANDRA-2400: Attachment: MCASSANDRA-tweaks.patch Patch to solve the issue Resolve Maven Ant Tasks from local Maven repository if present -- Key: CASSANDRA-2400 URL: https://issues.apache.org/jira/browse/CASSANDRA-2400 Project: Cassandra Issue Type: Improvement Affects Versions: 0.7.4 Reporter: Stephen Connolly Priority: Minor Fix For: 0.7.5 Attachments: MCASSANDRA-tweaks.patch To aid with testing using newer versions of Maven ANT Tasks it can be helpful to copy the version from the local repository if present rather than always downloading -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2400) Resolve Maven Ant Tasks from local Maven repository if present
[ https://issues.apache.org/jira/browse/CASSANDRA-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Connolly updated CASSANDRA-2400: Affects Version/s: 0.7.4 Fix Version/s: 0.7.5 Resolve Maven Ant Tasks from local Maven repository if present -- Key: CASSANDRA-2400 URL: https://issues.apache.org/jira/browse/CASSANDRA-2400 Project: Cassandra Issue Type: Improvement Affects Versions: 0.7.4 Reporter: Stephen Connolly Priority: Minor Fix For: 0.7.5 Attachments: MCASSANDRA-tweaks.patch To aid with testing using newer versions of Maven ANT Tasks it can be helpful to copy the version from the local repository if present rather than always downloading -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2400) Resolve Maven Ant Tasks from local Maven repository if present
[ https://issues.apache.org/jira/browse/CASSANDRA-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Connolly updated CASSANDRA-2400: Component/s: Packaging Resolve Maven Ant Tasks from local Maven repository if present -- Key: CASSANDRA-2400 URL: https://issues.apache.org/jira/browse/CASSANDRA-2400 Project: Cassandra Issue Type: Improvement Components: Packaging Affects Versions: 0.7.4 Reporter: Stephen Connolly Priority: Minor Fix For: 0.7.5 Attachments: MCASSANDRA-tweaks.patch To aid with testing using newer versions of Maven ANT Tasks it can be helpful to copy the version from the local repository if present rather than always downloading -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2398) Type specific compression
[
https://issues.apache.org/jira/browse/CASSANDRA-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012251#comment-13012251
]
Stu Hood edited comment on CASSANDRA-2398 at 3/29/11 12:31 AM:
---
Actually, one restriction we should put on the output blob is that it should
start with a length for skipability.
EDIT: rather than requiring a length at the front, could just add skip(version,
DataInput) method which parallels decompress, since some implementations (fixed
length values) will be able to skip efficiently without storing a length.
was (Author: stuhood):
Actually, one restriction we should put on the output blob is that it
should start with a length for skipability.
Type specific compression
-
Key: CASSANDRA-2398
URL: https://issues.apache.org/jira/browse/CASSANDRA-2398
Project: Cassandra
Issue Type: New Feature
Components: Core
Affects Versions: 0.8
Reporter: Stu Hood
Labels: compression
Attachments: example.diff
Cassandra has a lot of locations that are ripe for type specific compression.
A short list:
Indexes
* Keys compressed as BytesType, which could default to LZO/LZMA
* Offsets (delta and varint encoding)
* Column names added by 2319
Data
* Keys, columns, timestamps: see
http://wiki.apache.org/cassandra/FileFormatDesignDoc
A basic interface for type specific compression could be as simple as:
{code:java}
public void compress(int version, IteratorByteBuffer from, int count,
DataOutput to) throws IOException
public void decompress(int version, DataInput from, ListByteBuffer to)
throws IOException
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2387) Create a pig loadstorecaster for the loadfunc/storefunc
[ https://issues.apache.org/jira/browse/CASSANDRA-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna updated CASSANDRA-2387: Attachment: 2387-1.txt The beginnings of getting it to work with packed data. Added a call to get the schema through thrift when the pig load is performed. Then it uses the udf context to get the schema when it needs to in columnToTuple (as part of getNext). Create a pig loadstorecaster for the loadfunc/storefunc --- Key: CASSANDRA-2387 URL: https://issues.apache.org/jira/browse/CASSANDRA-2387 Project: Cassandra Issue Type: Improvement Reporter: Jeremy Hanna Assignee: Jeremy Hanna Labels: contrib, hadoop, pig Attachments: 2387-1.txt, loadstorecaster-patch.txt It appears that after we have applied validators to columns in the metadata that those column values appear as binary data when returned to pig. Validators pack values to bytebuffers to the appropriate width, so that seems to be the issue. So to address this, we should be able to create a loadstorecaster to convert cassandra data types to/from things that are readable by pig. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2387) Make it possible for pig to understand packed data
[ https://issues.apache.org/jira/browse/CASSANDRA-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna updated CASSANDRA-2387: Description: Packed values are throwing off pig. This ticket is to make it so pig can interpret packed values. Originally we thought we could just use a loadcaster. However, the only way we know how we can do it now is to get the schema through thrift and essentially perform the function of the loadcaster in the getNext method. (was: It appears that after we have applied validators to columns in the metadata that those column values appear as binary data when returned to pig. Validators pack values to bytebuffers to the appropriate width, so that seems to be the issue. So to address this, we should be able to create a loadstorecaster to convert cassandra data types to/from things that are readable by pig.) Summary: Make it possible for pig to understand packed data (was: Create a pig loadstorecaster for the loadfunc/storefunc) Make it possible for pig to understand packed data -- Key: CASSANDRA-2387 URL: https://issues.apache.org/jira/browse/CASSANDRA-2387 Project: Cassandra Issue Type: Improvement Reporter: Jeremy Hanna Assignee: Jeremy Hanna Labels: contrib, hadoop, pig Attachments: 2387-1.txt, loadstorecaster-patch.txt Packed values are throwing off pig. This ticket is to make it so pig can interpret packed values. Originally we thought we could just use a loadcaster. However, the only way we know how we can do it now is to get the schema through thrift and essentially perform the function of the loadcaster in the getNext method. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2387) Make it possible for pig to understand packed data
[ https://issues.apache.org/jira/browse/CASSANDRA-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012300#comment-13012300 ] Jeremy Hanna edited comment on CASSANDRA-2387 at 3/29/11 1:00 AM: -- The beginnings of getting it to work with packed data. Added a call to get the schema through thrift when the pig load is performed. Then it uses the udf context to get the schema when it needs to in columnToTuple (as part of getNext). With this patch: * only column values of type UTF8, int, and long are converted * the comparator class for column names is not currently converted * super columns' subcolumns do not deal with packed data was (Author: jeromatron): The beginnings of getting it to work with packed data. Added a call to get the schema through thrift when the pig load is performed. Then it uses the udf context to get the schema when it needs to in columnToTuple (as part of getNext). Make it possible for pig to understand packed data -- Key: CASSANDRA-2387 URL: https://issues.apache.org/jira/browse/CASSANDRA-2387 Project: Cassandra Issue Type: Improvement Reporter: Jeremy Hanna Assignee: Jeremy Hanna Labels: contrib, hadoop, pig Attachments: 2387-1.txt, loadstorecaster-patch.txt Packed values are throwing off pig. This ticket is to make it so pig can interpret packed values. Originally we thought we could just use a loadcaster. However, the only way we know how we can do it now is to get the schema through thrift and essentially perform the function of the loadcaster in the getNext method. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2398) Type specific compression
[
https://issues.apache.org/jira/browse/CASSANDRA-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stu Hood updated CASSANDRA-2398:
Description:
Cassandra has a lot of locations that are ripe for type specific compression. A
short list:
Indexes
* Keys compressed as BytesType, which could default to LZO/LZMA
* Offsets (delta and varint encoding)
* Column names added by 2319
Data
* Keys, columns, timestamps: see
http://wiki.apache.org/cassandra/FileFormatDesignDoc
A basic interface for type specific compression could be as simple as:
{code:java}
public void compress(int version, IteratorByteBuffer from, int count,
DataOutput to) throws IOException
public void decompress(int version, DataInput from, ListByteBuffer to) throws
IOException
public void skip(int version, DataInput from) throws IOException
{code}
was:
Cassandra has a lot of locations that are ripe for type specific compression. A
short list:
Indexes
* Keys compressed as BytesType, which could default to LZO/LZMA
* Offsets (delta and varint encoding)
* Column names added by 2319
Data
* Keys, columns, timestamps: see
http://wiki.apache.org/cassandra/FileFormatDesignDoc
A basic interface for type specific compression could be as simple as:
{code:java}
public void compress(int version, IteratorByteBuffer from, int count,
DataOutput to) throws IOException
public void decompress(int version, DataInput from, ListByteBuffer to) throws
IOException
{code}
Type specific compression
-
Key: CASSANDRA-2398
URL: https://issues.apache.org/jira/browse/CASSANDRA-2398
Project: Cassandra
Issue Type: New Feature
Components: Core
Affects Versions: 0.8
Reporter: Stu Hood
Labels: compression
Attachments: example.diff
Cassandra has a lot of locations that are ripe for type specific compression.
A short list:
Indexes
* Keys compressed as BytesType, which could default to LZO/LZMA
* Offsets (delta and varint encoding)
* Column names added by 2319
Data
* Keys, columns, timestamps: see
http://wiki.apache.org/cassandra/FileFormatDesignDoc
A basic interface for type specific compression could be as simple as:
{code:java}
public void compress(int version, IteratorByteBuffer from, int count,
DataOutput to) throws IOException
public void decompress(int version, DataInput from, ListByteBuffer to)
throws IOException
public void skip(int version, DataInput from) throws IOException
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2398) Type specific compression
[
https://issues.apache.org/jira/browse/CASSANDRA-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stu Hood updated CASSANDRA-2398:
Comment: was deleted
(was: Actually, one restriction we should put on the output blob is that it
should start with a length for skipability.
EDIT: rather than requiring a length at the front, could just add skip(version,
DataInput) method which parallels decompress, since some implementations (fixed
length values) will be able to skip efficiently without storing a length.)
Type specific compression
-
Key: CASSANDRA-2398
URL: https://issues.apache.org/jira/browse/CASSANDRA-2398
Project: Cassandra
Issue Type: New Feature
Components: Core
Affects Versions: 0.8
Reporter: Stu Hood
Labels: compression
Attachments: example.diff
Cassandra has a lot of locations that are ripe for type specific compression.
A short list:
Indexes
* Keys compressed as BytesType, which could default to LZO/LZMA
* Offsets (delta and varint encoding)
* Column names added by 2319
Data
* Keys, columns, timestamps: see
http://wiki.apache.org/cassandra/FileFormatDesignDoc
A basic interface for type specific compression could be as simple as:
{code:java}
public void compress(int version, IteratorByteBuffer from, int count,
DataOutput to) throws IOException
public void decompress(int version, DataInput from, ListByteBuffer to)
throws IOException
public void skip(int version, DataInput from) throws IOException
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2311) type validated row keys
[
https://issues.apache.org/jira/browse/CASSANDRA-2311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Evans updated CASSANDRA-2311:
--
Attachment: v1-0001-CASSANDRA-2311-missed-CFM-conversion.txt
type validated row keys
---
Key: CASSANDRA-2311
URL: https://issues.apache.org/jira/browse/CASSANDRA-2311
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Eric Evans
Assignee: Jon Hermes
Labels: cql
Fix For: 0.8
Attachments: 2311.txt,
v1-0001-CASSANDRA-2311-missed-CFM-conversion.txt
The idea here is to allow the assignment of a column-family-wide key type
used to perform validation, (similar to how default_validation_class does for
column values).
This should be as straightforward as extending the column family schema to
include the new attribute, and updating {{ThriftValidation.validateKey}} to
validate the key ({{AbstractType.validate}}).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (CASSANDRA-2311) type validated row keys
[
https://issues.apache.org/jira/browse/CASSANDRA-2311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Evans reopened CASSANDRA-2311:
---
Found a nit. {{key_validation_class}} was missed on one of the convertTo
methods.
Patch attached.
type validated row keys
---
Key: CASSANDRA-2311
URL: https://issues.apache.org/jira/browse/CASSANDRA-2311
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Eric Evans
Assignee: Jon Hermes
Labels: cql
Fix For: 0.8
Attachments: 2311.txt,
v1-0001-CASSANDRA-2311-missed-CFM-conversion.txt
The idea here is to allow the assignment of a column-family-wide key type
used to perform validation, (similar to how default_validation_class does for
column values).
This should be as straightforward as extending the column family schema to
include the new attribute, and updating {{ThriftValidation.validateKey}} to
validate the key ({{AbstractType.validate}}).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
