[jira] Updated: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1714: --- Attachment: CASSANDRA-1714-v6-0.7.patch zero-copy reads --- Key: CASSANDRA-1714 URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: 1714-v5-07.txt, CASSANDRA-1714-0.7.patch, CASSANDRA-1714-v2.patch, CASSANDRA-1714-v3.patch, CASSANDRA-1714-v4-trunk.patch, CASSANDRA-1714-v6-0.7.patch, CASSANDRA-1714.patch, zerocopy.txt Original Estimate: 32h Time Spent: 32h Remaining Estimate: 0h Since we are already using mmap'd ByteBuffers in MappedFileDataInput we should be able to do zero-copy reads (via buffer.slice()), which would give us better performance than CASSANDRA-1651 without having to worry about buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1472) Add bitmap secondary indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981758#action_12981758 ] Gary Dusbabek commented on CASSANDRA-1472: -- bq. We can re-implement a file format identical to the specification... That's not so different than rolling our own serialization. At that point, we've gained nothing but interop (how much of a goal is that? If it is, it needs to be stated) by using avro structures and serialization. Add bitmap secondary indexes Key: CASSANDRA-1472 URL: https://issues.apache.org/jira/browse/CASSANDRA-1472 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Fix For: 0.7.1 Attachments: 0.7-1472-v5.tgz, 0.7-1472-v6.tgz, 0019-Rename-bugfixes-and-fileclose.txt, 1472-v3.tgz, 1472-v4.tgz, 1472-v5.tgz, anatomy.png, v4-bench-c32.txt Bitmap indexes are a very efficient structure for dealing with immutable data. We can take advantage of the fact that SSTables are immutable by attaching them directly to SSTables as a new component (supported by CASSANDRA-1471). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1985) read repair on CL.ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981846#action_12981846 ] Kelvin Kakugawa commented on CASSANDRA-1985: Yes, you're right, that does schedule it, once. The process for CL.ONE is: 1) schedule RR for data+digest and watch for a DigestMismatchException, 2) catch DME and call repair() to do a RR for data-only. However, the handler for the second RR (that repair() returns) is never used. So, even though it's collecting all the data repair messages, the RRR's resolve() never gets called. read repair on CL.ONE regression Key: CASSANDRA-1985 URL: https://issues.apache.org/jira/browse/CASSANDRA-1985 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0, 0.7.1, 0.8 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1, 0.8 Attachments: CASSANDRA-1985-0001-fix-CL.ONE-read-repair-regression.patch read repair w/ CL.ONE had a regression. The RepairCallback was dropped (in the background for CL.ONE), so ReadResponseResolver : resolve() was never called. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CASSANDRA-1986) write CL ONE regression
write CL ONE regression - Key: CASSANDRA-1986 URL: https://issues.apache.org/jira/browse/CASSANDRA-1986 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0, 0.7.1, 0.8 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1, 0.8, 0.7.0 write CL ONE regression by the DC refactor. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1986) write CL ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kelvin Kakugawa updated CASSANDRA-1986: --- Attachment: CASSANDRA-1986-0001-fix-write-CL-ONE-regression.patch fix regression: send local DC messages to the follower nodes, as well. write CL ONE regression - Key: CASSANDRA-1986 URL: https://issues.apache.org/jira/browse/CASSANDRA-1986 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0, 0.7.1, 0.8 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.0, 0.7.1, 0.8 Attachments: CASSANDRA-1986-0001-fix-write-CL-ONE-regression.patch write CL ONE regression by the DC refactor. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1985) read repair on CL.ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981874#action_12981874 ] Jonathan Ellis commented on CASSANDRA-1985: --- The second RR is this one: {code} RepairCallbackRow handler = repair(command, endpoints); ... repairResponseHandlers.add(handler); ... for (RepairCallbackRow handler : repairResponseHandlers) { try { Row row = handler.get(); {code} read repair on CL.ONE regression Key: CASSANDRA-1985 URL: https://issues.apache.org/jira/browse/CASSANDRA-1985 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0, 0.7.1, 0.8 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1, 0.8 Attachments: CASSANDRA-1985-0001-fix-CL.ONE-read-repair-regression.patch read repair w/ CL.ONE had a regression. The RepairCallback was dropped (in the background for CL.ONE), so ReadResponseResolver : resolve() was never called. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-1985) read repair on CL.ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981874#action_12981874 ] Jonathan Ellis edited comment on CASSANDRA-1985 at 1/14/11 1:51 PM: The second RR (that is, the second read request, for performing repair when a mismatch was detected by the digest read) is this one: {code} RepairCallbackRow handler = repair(command, endpoints); ... repairResponseHandlers.add(handler); ... for (RepairCallbackRow handler : repairResponseHandlers) { try { Row row = handler.get(); {code} was (Author: jbellis): The second RR is this one: {code} RepairCallbackRow handler = repair(command, endpoints); ... repairResponseHandlers.add(handler); ... for (RepairCallbackRow handler : repairResponseHandlers) { try { Row row = handler.get(); {code} read repair on CL.ONE regression Key: CASSANDRA-1985 URL: https://issues.apache.org/jira/browse/CASSANDRA-1985 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0, 0.7.1, 0.8 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1, 0.8 Attachments: CASSANDRA-1985-0001-fix-CL.ONE-read-repair-regression.patch read repair w/ CL.ONE had a regression. The RepairCallback was dropped (in the background for CL.ONE), so ReadResponseResolver : resolve() was never called. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
svn commit: r1059124 - /cassandra/branches/cassandra-0.7/conf/cassandra.yaml
Author: jbellis Date: Fri Jan 14 19:13:44 2011 New Revision: 1059124 URL: http://svn.apache.org/viewvc?rev=1059124view=rev Log: update concurrent_reads default setting + comments patch by Stu Hood; reviewed by jbellis for CASSANDRA-1972 Modified: cassandra/branches/cassandra-0.7/conf/cassandra.yaml Modified: cassandra/branches/cassandra-0.7/conf/cassandra.yaml URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/conf/cassandra.yaml?rev=1059124r1=1059123r2=1059124view=diff == --- cassandra/branches/cassandra-0.7/conf/cassandra.yaml (original) +++ cassandra/branches/cassandra-0.7/conf/cassandra.yaml Fri Jan 14 19:13:44 2011 @@ -104,12 +104,16 @@ seeds: # non-mmapped i/o.) disk_access_mode: auto -# Unlike most systems, in Cassandra writes are faster than reads, so -# you can afford more of those in parallel. A good rule of thumb is 2 -# concurrent reads per processor core. Increase ConcurrentWrites to -# the number of clients writing at once if you enable CommitLogSync + -# CommitLogSyncDelay. -- -concurrent_reads: 8 +# For workloads with more data than can fit in memory, Cassandra's +# bottleneck will be reads that need to fetch data from +# disk. concurrent_reads should be set to (16 * number_of_drives) in +# order to allow the operations to enqueue low enough in the stack +# that the OS and drives can reorder them. +# +# On the other hand, since writes are almost never IO bound, the ideal +# number of concurrent_writes is dependent on the number of cores in +# your system; (8 * number_of_cores) is a good rule of thumb. +concurrent_reads: 32 concurrent_writes: 32 # This sets the amount of memtable flush writer threads. These will
[jira] Commented: (CASSANDRA-1935) Refuse to open SSTables from the future
[ https://issues.apache.org/jira/browse/CASSANDRA-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981885#action_12981885 ] Jonathan Ellis commented on CASSANDRA-1935: --- Any reason not to commit this to 0.7 branch too? Refuse to open SSTables from the future --- Key: CASSANDRA-1935 URL: https://issues.apache.org/jira/browse/CASSANDRA-1935 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Priority: Minor Fix For: 0.8 Attachments: CASSANDRA-1935.patch If somebody has rolled back to a previous version of Cassandra that is unable to read an SSTable written by a future version correctly (indicated by a version change), failing fast is safer than accidentally performing a compaction that rewrites incorrect data and leaves you in an odd state. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1985) read repair on CL.ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981898#action_12981898 ] Kelvin Kakugawa commented on CASSANDRA-1985: Yes, that's correct for read CL ONE. A quorum / all read goes through that path. However, the CL.ONE case does not go through that code path. The branch in the code is in fetchRows(...) when it checks for randomlyReadRepair(...). If the targets handler.blockfor, it does a background repair via RepairRunner in service.StorageProxy. i.e. it won't go through the block of code you pasted, because a DigestMismatchException won't be thrown for CL.ONE. Now, let's look at RepairRunner : runMayThrow. It calls repair(command, endpoints), but the RepairCallbackrow that is returned by repair(...) is dropped on the floor. So, resolve is never called on that RepairCallback's ReadResponseResolver. The above error was found via my own set of distributed tests. read repair on CL.ONE regression Key: CASSANDRA-1985 URL: https://issues.apache.org/jira/browse/CASSANDRA-1985 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0, 0.7.1, 0.8 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1, 0.8 Attachments: CASSANDRA-1985-0001-fix-CL.ONE-read-repair-regression.patch read repair w/ CL.ONE had a regression. The RepairCallback was dropped (in the background for CL.ONE), so ReadResponseResolver : resolve() was never called. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CASSANDRA-1979) CassandraServiceDataCleaner.prepare() fails with IOException.
[ https://issues.apache.org/jira/browse/CASSANDRA-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-1979: - Assignee: Nate McCall CassandraServiceDataCleaner.prepare() fails with IOException. - Key: CASSANDRA-1979 URL: https://issues.apache.org/jira/browse/CASSANDRA-1979 Project: Cassandra Issue Type: Bug Components: Contrib Affects Versions: 0.7.0 Environment: Windows XP Reporter: Chris Ng Assignee: Nate McCall CassandraServiceDataCleaner.prepare() fails with an IOException if run in isolation. It seems that initializing the DataDescriptor creates a new CommitLog file, and then the cleaner tries to delete this file and fails. 16:06:07.204 [main] INFO o.a.c.config.DatabaseDescriptor - Loading settings from file:/C:/workspace/sandbox/target/classes/cassandra.yaml 16:06:07.282 [main] DEBUG o.a.c.config.DatabaseDescriptor - Syncing log with a period of 1 16:06:07.282 [main] INFO o.a.c.config.DatabaseDescriptor - DiskAccessMode 'auto' determined to be standard, indexAccessMode is standard 16:06:07.797 [main] DEBUG o.a.c.config.DatabaseDescriptor - setting auto_bootstrap to false 16:06:07.797 [main] INFO o.a.c.db.commitlog.CommitLogSegment - Creating new commitlog segment target/var/lib/cassandra/commitlog\CommitLog-1294934767797.log 16:06:07.813 [main] DEBUG o.apache.cassandra.io.util.FileUtils - Deleting CommitLog-1294934767797.log Exception in thread main java.io.IOException: Failed to delete C:\workspace\sandbox\target\var\lib\cassandra\commitlog\CommitLog-1294934767797.log at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:54) at org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:201) at org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner.cleanDir(CassandraServiceDataCleaner.java:99) at org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner.cleanupDataDirectories(CassandraServiceDataCleaner.java:53) at org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner.prepare(CassandraServiceDataCleaner.java:44) at cng.sandbox.App.main(App.java:15) This also seems to leave a bunch of threads running in the background, so the process has to be manually killed. This was tested with the javautils in the 0.7.0 branch. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1985) read repair on CL.ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981900#action_12981900 ] Jonathan Ellis commented on CASSANDRA-1985: --- I get it now: the callback from the repair() call in RepairRunner is the one that we don't resolve. read repair on CL.ONE regression Key: CASSANDRA-1985 URL: https://issues.apache.org/jira/browse/CASSANDRA-1985 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0, 0.7.1, 0.8 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1, 0.8 Attachments: CASSANDRA-1985-0001-fix-CL.ONE-read-repair-regression.patch read repair w/ CL.ONE had a regression. The RepairCallback was dropped (in the background for CL.ONE), so ReadResponseResolver : resolve() was never called. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1972) Default concurrency values are improperly proportioned
[ https://issues.apache.org/jira/browse/CASSANDRA-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981903#action_12981903 ] Hudson commented on CASSANDRA-1972: --- Integrated in Cassandra-0.7 #157 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/157/]) update concurrent_reads default setting + comments patch by Stu Hood; reviewed by jbellis for CASSANDRA-1972 Default concurrency values are improperly proportioned -- Key: CASSANDRA-1972 URL: https://issues.apache.org/jira/browse/CASSANDRA-1972 Project: Cassandra Issue Type: Bug Reporter: Stu Hood Assignee: Stu Hood Priority: Minor Fix For: 0.7.1 Attachments: 0001-Recommendation-changes.txt The default/suggested {{concurrent_reads}} value is much too low. It assumes that CPU will be the bottleneck, rather than IO, and for most deployments, this will not be the case. Additionally it is better to be queued for IO in the kernel or on your device than in user space, because the former work to optimize queue order. Additionally, reads are much cheaper than writes in terms of CPU time (since writes can experience contention due to retries), so while {{concurrent_writes}} should probably factor in the number of cores on the machine, {{concurrent_reads}} should probably be calculated purely by number of spindles. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1935) Refuse to open SSTables from the future
[ https://issues.apache.org/jira/browse/CASSANDRA-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981904#action_12981904 ] Ryan King commented on CASSANDRA-1935: -- Not that I can think of. Refuse to open SSTables from the future --- Key: CASSANDRA-1935 URL: https://issues.apache.org/jira/browse/CASSANDRA-1935 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Priority: Minor Fix For: 0.8 Attachments: CASSANDRA-1935.patch If somebody has rolled back to a previous version of Cassandra that is unable to read an SSTable written by a future version correctly (indicated by a version change), failing fast is safer than accidentally performing a compaction that rewrites incorrect data and leaves you in an odd state. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1964) MutationTest of the distributed-test suite fails
[ https://issues.apache.org/jira/browse/CASSANDRA-1964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1964: -- Reviewer: xedin MutationTest of the distributed-test suite fails Key: CASSANDRA-1964 URL: https://issues.apache.org/jira/browse/CASSANDRA-1964 Project: Cassandra Issue Type: Bug Components: Core Environment: OS X Version 10.6.6 java version 1.6.0_22 Java(TM) SE Runtime Environment (build 1.6.0_22-b04-307-10M3261) Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03-307, mixed mode) Apache Ant version 1.8.1 compiled on September 21 2010 Reporter: Michael Allen Assignee: Stu Hood Priority: Minor Fix For: 0.8 Attachments: 0001-Add-a-builder-for-a-retrying-get-attempt-and-use-to-im.txt MutationTest of the distributed-test test suite causes errors on trunk. To reproduce, issue: ant distributed-test -Dwhirr.config=path_to_whirr_config_file from the project root. relevant whirr configuration settings used: whirr.service-name=cassandra whirr.cluster-name=cassandra_test whirr.instance-templates=4 cassandra whirr.version=0.3.0-incubating-SNAPSHOT whirr.location-id=us-west-1 whirr.image-id=us-west-1/ami-16f3a253 whirr.hardware-id=m1.large whirr.blobstore.provider=s3 whirr.blobstore.container=tawamuducassandratests whirr.provider=ec2 whirr.run-url-base=http://hoodidge.net/scripts/ Traceback: distributed-test: [echo] running distributed tests [junit] WARNING: multiple versions of ant detected in path for junit [junit] jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class [junit] and jar:file:/Users/mallen/Desktop/cassandra-trunk/build/lib/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class [junit] Testsuite: org.apache.cassandra.MovementTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 446.65 sec [junit] [junit] - Standard Error - [junit] SLF4J: Class path contains multiple SLF4J bindings. [junit] SLF4J: Found binding in [jar:file:/Users/mallen/Desktop/cassandra-trunk/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit] SLF4J: Found binding in [jar:file:/Users/mallen/Desktop/cassandra-trunk/build/test/lib/jars/whirr-cli-0.3.0-incubating-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. [junit] WARN 12:12:46,654 over limit 471283/262144: wrote temp file [junit] WARN 12:12:48,572 over limit 374979/262144: wrote temp file [junit] WARN 12:12:50,701 over limit 892174/262144: wrote temp file [junit] WARN 12:12:54,442 over limit 612358/262144: wrote temp file [junit] - --- [junit] Testsuite: org.apache.cassandra.MutationTest [junit] Tests run: 4, Failures: 0, Errors: 3, Time elapsed: 110.971 sec [junit] [junit] Testcase: testInsert(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] NotFoundException() [junit] at org.apache.cassandra.thrift.Cassandra$get_result.read(Cassandra.java:6900) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:568) [junit] at org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:541) [junit] at org.apache.cassandra.MutationTest.getColumn(MutationTest.java:210) [junit] at org.apache.cassandra.MutationTest.testInsert(MutationTest.java:66) [junit] [junit] [junit] Testcase: testWriteAllReadOne(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] NotFoundException() [junit] at org.apache.cassandra.thrift.Cassandra$get_result.read(Cassandra.java:6900) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:568) [junit] at org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:541) [junit] at org.apache.cassandra.MutationTest.getColumn(MutationTest.java:210) [junit] at org.apache.cassandra.MutationTest.testWriteAllReadOne(MutationTest.java:87) [junit] [junit] [junit] Testcase: testWriteOneReadAll(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] TimedOutException() [junit] at org.apache.cassandra.thrift.Cassandra$insert_result.read(Cassandra.java:15392) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_insert(Cassandra.java:907) [junit] at org.apache.cassandra.thrift.Cassandra$Client.insert(Cassandra.java:879) [junit] at
svn commit: r1059153 - in /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service: RepairCallback.java StorageProxy.java
Author: jbellis Date: Fri Jan 14 20:17:26 2011 New Revision: 1059153 URL: http://svn.apache.org/viewvc?rev=1059153view=rev Log: add comments, remove obsolete TODO patch by jbellis Modified: cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/RepairCallback.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java Modified: cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/RepairCallback.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/RepairCallback.java?rev=1059153r1=1059152r2=1059153view=diff == --- cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/RepairCallback.java (original) +++ cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/RepairCallback.java Fri Jan 14 20:17:26 2011 @@ -51,5 +51,4 @@ public class RepairCallbackT implement if (resolver.getMessageCount() == endpoints.size()) condition.signal(); } - } Modified: cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java?rev=1059153r1=1059152r2=1059153view=diff == --- cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java (original) +++ cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java Fri Jan 14 20:17:26 2011 @@ -389,7 +389,7 @@ public class StorageProxy implements Sto try { long startTime2 = System.currentTimeMillis(); -row = readCallback.get(); +row = readCallback.get(); // CL.ONE is special cased here to ignore digests even if some have arrived if (row != null) rows.add(row); @@ -440,7 +440,6 @@ public class StorageProxy implements Sto return new ReadCallback(resolver, consistencyLevel, table); } -// TODO repair resolver shouldn't take consistencylevel (it should repair exactly as many as it receives replies for) private static RepairCallbackRow repair(ReadCommand command, ListInetAddress endpoints) throws IOException {
[jira] Updated: (CASSANDRA-1718) cassandra should chdir / when daemonizing
[ https://issues.apache.org/jira/browse/CASSANDRA-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Evans updated CASSANDRA-1718: -- Attachment: v2-0001-CASSANDRA-1718-chdir-on-startup.txt cassandra should chdir / when daemonizing - Key: CASSANDRA-1718 URL: https://issues.apache.org/jira/browse/CASSANDRA-1718 Project: Cassandra Issue Type: Bug Components: Packaging Environment: Debian squeeze, Cassandra 0.7.0-beta3 and trunk (r1032649) Reporter: paul cannon Assignee: Eric Evans Priority: Minor Fix For: 0.7.1 Attachments: v1-0001-CASSANDRA-1718-switch-to-home-directory-on-startup.txt, v2-0001-CASSANDRA-1718-chdir-on-startup.txt Common practice when daemonizing is to cd / to avoid pinning a filesystem. For example, if the oper happens to start Cassandra (by itself, or with a manual jsvc invocation, or with the initscript) in /mnt/usb-storage, and there is something mounted there, then the oper will not be able to unmount the usb device that was mounted at that location, since the cassandra process has it open as its cwd. evidence that this isn't being done already: {noformat} ~% sudo lsof -p 9775 | awk '$4==cwd' jsvc9775 cassandra cwdDIR8,1 4096 147675 /home/paul/packages/cassandra/trunk {noformat} (That instance was invoked using the Debian initscript.) Obviously chdir(/) isn't necessary when not daemonizing, although it shouldn't hurt either. If there are concerns about Cassandra having an ongoing ability to open filenames relative to its original working directory, then it should be sufficient just to do a cd / in the initscript before starting Cassandra. That case, at least, is particularly important. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1985) read repair on CL.ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1985: -- Attachment: 1985-v2.txt v2 keeps the resolve off the response stage, which we want to keep very low latency. read repair on CL.ONE regression Key: CASSANDRA-1985 URL: https://issues.apache.org/jira/browse/CASSANDRA-1985 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0, 0.7.1, 0.8 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1, 0.8 Attachments: 1985-v2.txt, CASSANDRA-1985-0001-fix-CL.ONE-read-repair-regression.patch read repair w/ CL.ONE had a regression. The RepairCallback was dropped (in the background for CL.ONE), so ReadResponseResolver : resolve() was never called. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-926) remove alternative RPC interface using Avro
[ https://issues.apache.org/jira/browse/CASSANDRA-926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Evans updated CASSANDRA-926: - Attachment: v2-0006-move-remaining-avro-records-to-hadoop-package.txt v2-0005-remove-avro-rpc-source.txt v2-0004-remove-avro-functional-tests.txt v2-0003-remove-option-to-launch-avro-rpc-daemon.txt v2-0002-don-t-build-schema-for-rpc-interface.txt v2-0001-CASSANDRA-926-denormalize-avro-internode-rpc-schema.txt remove alternative RPC interface using Avro --- Key: CASSANDRA-926 URL: https://issues.apache.org/jira/browse/CASSANDRA-926 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Eric Evans Assignee: Eric Evans Priority: Minor Attachments: v1-0001-CASSANDRA-926-denormalize-avro-internode-rpc-schema.txt, v1-0002-don-t-build-schema-for-rpc-interface.txt, v1-0003-remove-option-to-launch-avro-rpc-daemon.txt, v1-0004-remove-avro-functional-tests.txt, v1-0005-remove-avro-rpc-source.txt, v1-0006-move-remaining-avro-records-to-hadoop-package.txt, v2-0001-CASSANDRA-926-denormalize-avro-internode-rpc-schema.txt, v2-0002-don-t-build-schema-for-rpc-interface.txt, v2-0003-remove-option-to-launch-avro-rpc-daemon.txt, v2-0004-remove-avro-functional-tests.txt, v2-0005-remove-avro-rpc-source.txt, v2-0006-move-remaining-avro-records-to-hadoop-package.txt Original Estimate: 0h Remaining Estimate: 0h *UPDATE: This ticket is now for the removal of the Avro interface, (see http://thread.gmane.org/gmane.comp.db.cassandra.client.devel/36 for background).* Avro is data serialization and RPC framework similar to Thrift. It provides: * Rich data structures. * A compact, fast, binary data format. * A container file, to store persistent data. * Remote procedure call (RPC). * Simple integration with dynamic languages. Code generation is not required to read or write data files nor to use or implement RPC protocols. Code generation as an optional optimization, only worth implementing for statically typed languages. Cassandra's Avro interface is being structured in a way that closely mirrors the existing Thrift interface, both in terms of public facing API, and how it is implemented. GSOC students interested in this task should begin by familiarizing themselves with Cassandra's Thrift service (org.apache.cassandra.thrift). Note: This is a very large and long-running task so treat this as a meta-issue and add sub-tasks and/or blocking issues as appropriate. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
svn commit: r1059160 - /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/RetryingScheduledThreadPoolExecutor.java
Author: jbellis Date: Fri Jan 14 20:46:35 2011 New Revision: 1059160 URL: http://svn.apache.org/viewvc?rev=1059160view=rev Log: call Thread.getDefaultUncaughtExceptionHandler instead of logger.error in on scheduled task failure patch by jbellis Modified: cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/RetryingScheduledThreadPoolExecutor.java Modified: cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/RetryingScheduledThreadPoolExecutor.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/RetryingScheduledThreadPoolExecutor.java?rev=1059160r1=1059159r2=1059160view=diff == --- cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/RetryingScheduledThreadPoolExecutor.java (original) +++ cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/RetryingScheduledThreadPoolExecutor.java Fri Jan 14 20:46:35 2011 @@ -27,8 +27,6 @@ import org.apache.log4j.Logger; public class RetryingScheduledThreadPoolExecutor extends ScheduledThreadPoolExecutor { -protected static Logger logger = Logger.getLogger(RetryingScheduledThreadPoolExecutor.class); - public RetryingScheduledThreadPoolExecutor(String threadPoolName, int priority) { this(1, threadPoolName, priority); @@ -50,6 +48,11 @@ public class RetryingScheduledThreadPool return new LoggingScheduledFutureV(task); } +/** + * Wraps RunnableScheduledFuture.run to log an error on exception rather than kill the executor thread. + * All the other methods just wrap the RSF counterpart. + * @param V + */ private static class LoggingScheduledFutureV implements RunnableScheduledFutureV { private final RunnableScheduledFutureV task; @@ -82,7 +85,8 @@ public class RetryingScheduledThreadPool } catch (Exception e) { -logger.error(error running scheduled task, e); +if (Thread.getDefaultUncaughtExceptionHandler() != null) + Thread.getDefaultUncaughtExceptionHandler().uncaughtException(Thread.currentThread(), e.getCause()); } }
[jira] Commented: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981923#action_12981923 ] T Jake Luciani commented on CASSANDRA-1714: --- latest patch looks right, tested row cache before and after compaction, all ok. +1. I'll commit tonight unless someone chimes in with anything else zero-copy reads --- Key: CASSANDRA-1714 URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: 1714-v5-07.txt, CASSANDRA-1714-0.7.patch, CASSANDRA-1714-v2.patch, CASSANDRA-1714-v3.patch, CASSANDRA-1714-v4-trunk.patch, CASSANDRA-1714-v6-0.7.patch, CASSANDRA-1714.patch, zerocopy.txt Original Estimate: 32h Time Spent: 32h Remaining Estimate: 0h Since we are already using mmap'd ByteBuffers in MappedFileDataInput we should be able to do zero-copy reads (via buffer.slice()), which would give us better performance than CASSANDRA-1651 without having to worry about buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1986) write CL ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1986: -- Affects Version/s: (was: 0.7.0) (was: 0.8) Fix Version/s: (was: 0.7.0) (was: 0.8) write CL ONE regression - Key: CASSANDRA-1986 URL: https://issues.apache.org/jira/browse/CASSANDRA-1986 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.1 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1 Attachments: CASSANDRA-1986-0001-fix-write-CL-ONE-regression.patch write CL ONE regression by the DC refactor. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1985) read repair on CL.ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1985: -- Affects Version/s: (was: 0.7.0) (was: 0.8) Fix Version/s: (was: 0.8) read repair on CL.ONE regression Key: CASSANDRA-1985 URL: https://issues.apache.org/jira/browse/CASSANDRA-1985 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.1 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1 Attachments: 1985-v2.txt, CASSANDRA-1985-0001-fix-CL.ONE-read-repair-regression.patch read repair w/ CL.ONE had a regression. The RepairCallback was dropped (in the background for CL.ONE), so ReadResponseResolver : resolve() was never called. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[Cassandra Wiki] Update of HowToContribute by JonathanEllis
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The HowToContribute page has been changed by JonathanEllis. The comment on this change is: add section on svn merging. http://wiki.apache.org/cassandra/HowToContribute?action=diffrev1=33rev2=34 -- 1. If not, create a ticket describing the change you're proposing in the issue tracker 1. Check out the latest version of the source code * svn checkout http://svn.apache.org/repos/asf/cassandra/trunk cassandra-trunk - or +. or * git clone git://git.apache.org/cassandra.git 1. Modify the source to include the improvement/bugfix * Verify that you follow Cassandra's CodeStyle. @@ -37, +37 @@ ''Note: If you're setup with both the Thrift and Avro dependencies, you can run all of the tests by simply typing `nosetests` from the top-level directory'' === Running the code coverage task === - 1. Unzip this one: http://sourceforge.net/projects/cobertura/files/cobertura/1.9.4.1/cobertura-1.9.4.1-bin.zip/download 1. `ant codecoverage -Dcobertura.dir=/path/to/cobertura` 1. `/path/to/cobertura/cobertura-report.sh --destination build/cobertura/html source code src/java` @@ -49, +48 @@ == IDE == * [[RunningCassandraInIDEA]] - * [[RunningCassandraInEclipse]] + * RunningCassandraInEclipse * IntelliJ Project Settings: * Main Class for Thrift: org.apache.cassandra.thrift.CassandraDaemon (Used to be org.apache.cassandra.service.CassandraDaemon ) * Main Class for Avro: org.apache.cassandra.avro.CassandraDaemon (Exprimental, introduced in 0.6) @@ -99, +98 @@ git config svn.authorsfile .git/authors.txt git svn init --prefix=origin/ --branches=branches https://svn.apache.org/repos/asf/cassandra }}} + == Committing == + Got commit access? Outstanding! Here are the conventions we follow. + * Commit messages take the form of + {{{ + explanation + patch by author; reviewed by committer for CASSANDRA-ticket + }}} + * When committing to multiple branches, start with the most-stable and merge forwards. For instance, if you had a fix to apply to 0.6, 0.7, and trunk, you would first commit to 0.6. Then, from your 0.7 branch checkout, you would run svn merge https://svn.apache.org/repos/asf/cassandra/branches/cassandra-0.6;, resolve any conflicts, and commit. Then, from your trunk checkout, you would run svn merge https://svn.apache.org/repos/asf/cassandra/branches/cassandra-0.7;, again resolve conflicts, and commit. + + See http://video.google.com/videoplay?docid=-577744660535947210 for an in-depth explanation of why fixes should be merged forwards from more-stable branches, rather than backported from trunk. + + This workflow also makes it so svn knows what commits have been made to earlier branches but not to trunk: if you forget to merge a fix immediately, the next time someone goes to merge from the branch, svn will incorporate the forgotten ones too. +
[jira] Commented: (CASSANDRA-1459) Allow modification of HintedHandoff configuration to be changed on the fly per node.
[ https://issues.apache.org/jira/browse/CASSANDRA-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981926#action_12981926 ] Brandon Williams commented on CASSANDRA-1459: - ISTM that a simple approach here would be to define a maximum amount of time to hint for, since tracking a node's downtime is easy and then you don't have to go poke a JMX on 100 nodes to disable HH for a dead host. Allow modification of HintedHandoff configuration to be changed on the fly per node. Key: CASSANDRA-1459 URL: https://issues.apache.org/jira/browse/CASSANDRA-1459 Project: Cassandra Issue Type: Improvement Components: Tools Environment: N/A Reporter: Jeff Davey Assignee: Brandon Williams Fix For: 0.7.1 Original Estimate: 6h Remaining Estimate: 6h If there is an extended downtime of a node, allow Hinted Handoff to be disabled specifically for that node rather than having to decommission it. benblack If a node will be down for an extended period, it would be useful to be able to disable HH for it until it returns, without having to reconfigure and restart nodes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[Cassandra Wiki] Trivial Update of HowToContribute by JonathanEllis
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The HowToContribute page has been changed by JonathanEllis. http://wiki.apache.org/cassandra/HowToContribute?action=diffrev1=34rev2=35 -- == Committing == Got commit access? Outstanding! Here are the conventions we follow. - * Commit messages take the form of + Commit messages take the form of {{{ explanation patch by author; reviewed by committer for CASSANDRA-ticket }}} + - * When committing to multiple branches, start with the most-stable and merge forwards. For instance, if you had a fix to apply to 0.6, 0.7, and trunk, you would first commit to 0.6. Then, from your 0.7 branch checkout, you would run svn merge https://svn.apache.org/repos/asf/cassandra/branches/cassandra-0.6;, resolve any conflicts, and commit. Then, from your trunk checkout, you would run svn merge https://svn.apache.org/repos/asf/cassandra/branches/cassandra-0.7;, again resolve conflicts, and commit. + When committing to multiple branches, start with the most-stable and merge forwards. For instance, if you had a fix to apply to 0.6, 0.7, and trunk, you would first commit to 0.6. Then, from your 0.7 branch checkout, you would run svn merge !https://svn.apache.org/repos/asf/cassandra/branches/cassandra-0.6;, resolve any conflicts, and commit. Then, from your trunk checkout, you would run svn merge !https://svn.apache.org/repos/asf/cassandra/branches/cassandra-0.7;, again resolve conflicts, and commit. See http://video.google.com/videoplay?docid=-577744660535947210 for an in-depth explanation of why fixes should be merged forwards from more-stable branches, rather than backported from trunk.
[jira] Commented: (CASSANDRA-1459) Allow modification of HintedHandoff configuration to be changed on the fly per node.
[ https://issues.apache.org/jira/browse/CASSANDRA-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981928#action_12981928 ] Jonathan Ellis commented on CASSANDRA-1459: --- +1 Allow modification of HintedHandoff configuration to be changed on the fly per node. Key: CASSANDRA-1459 URL: https://issues.apache.org/jira/browse/CASSANDRA-1459 Project: Cassandra Issue Type: Improvement Components: Tools Environment: N/A Reporter: Jeff Davey Assignee: Brandon Williams Fix For: 0.7.1 Original Estimate: 6h Remaining Estimate: 6h If there is an extended downtime of a node, allow Hinted Handoff to be disabled specifically for that node rather than having to decommission it. benblack If a node will be down for an extended period, it would be useful to be able to disable HH for it until it returns, without having to reconfigure and restart nodes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1964) MutationTest of the distributed-test suite fails
[ https://issues.apache.org/jira/browse/CASSANDRA-1964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-1964: Attachment: (was: 0001-Add-a-builder-for-a-retrying-get-attempt-and-use-to-im.txt) MutationTest of the distributed-test suite fails Key: CASSANDRA-1964 URL: https://issues.apache.org/jira/browse/CASSANDRA-1964 Project: Cassandra Issue Type: Bug Components: Core Environment: OS X Version 10.6.6 java version 1.6.0_22 Java(TM) SE Runtime Environment (build 1.6.0_22-b04-307-10M3261) Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03-307, mixed mode) Apache Ant version 1.8.1 compiled on September 21 2010 Reporter: Michael Allen Assignee: Stu Hood Priority: Minor Fix For: 0.8 Attachments: 0001-Add-a-builder-for-a-retrying-get-attempt-and-use-to-im.txt MutationTest of the distributed-test test suite causes errors on trunk. To reproduce, issue: ant distributed-test -Dwhirr.config=path_to_whirr_config_file from the project root. relevant whirr configuration settings used: whirr.service-name=cassandra whirr.cluster-name=cassandra_test whirr.instance-templates=4 cassandra whirr.version=0.3.0-incubating-SNAPSHOT whirr.location-id=us-west-1 whirr.image-id=us-west-1/ami-16f3a253 whirr.hardware-id=m1.large whirr.blobstore.provider=s3 whirr.blobstore.container=tawamuducassandratests whirr.provider=ec2 whirr.run-url-base=http://hoodidge.net/scripts/ Traceback: distributed-test: [echo] running distributed tests [junit] WARNING: multiple versions of ant detected in path for junit [junit] jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class [junit] and jar:file:/Users/mallen/Desktop/cassandra-trunk/build/lib/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class [junit] Testsuite: org.apache.cassandra.MovementTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 446.65 sec [junit] [junit] - Standard Error - [junit] SLF4J: Class path contains multiple SLF4J bindings. [junit] SLF4J: Found binding in [jar:file:/Users/mallen/Desktop/cassandra-trunk/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit] SLF4J: Found binding in [jar:file:/Users/mallen/Desktop/cassandra-trunk/build/test/lib/jars/whirr-cli-0.3.0-incubating-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. [junit] WARN 12:12:46,654 over limit 471283/262144: wrote temp file [junit] WARN 12:12:48,572 over limit 374979/262144: wrote temp file [junit] WARN 12:12:50,701 over limit 892174/262144: wrote temp file [junit] WARN 12:12:54,442 over limit 612358/262144: wrote temp file [junit] - --- [junit] Testsuite: org.apache.cassandra.MutationTest [junit] Tests run: 4, Failures: 0, Errors: 3, Time elapsed: 110.971 sec [junit] [junit] Testcase: testInsert(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] NotFoundException() [junit] at org.apache.cassandra.thrift.Cassandra$get_result.read(Cassandra.java:6900) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:568) [junit] at org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:541) [junit] at org.apache.cassandra.MutationTest.getColumn(MutationTest.java:210) [junit] at org.apache.cassandra.MutationTest.testInsert(MutationTest.java:66) [junit] [junit] [junit] Testcase: testWriteAllReadOne(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] NotFoundException() [junit] at org.apache.cassandra.thrift.Cassandra$get_result.read(Cassandra.java:6900) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:568) [junit] at org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:541) [junit] at org.apache.cassandra.MutationTest.getColumn(MutationTest.java:210) [junit] at org.apache.cassandra.MutationTest.testWriteAllReadOne(MutationTest.java:87) [junit] [junit] [junit] Testcase: testWriteOneReadAll(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] TimedOutException() [junit] at org.apache.cassandra.thrift.Cassandra$insert_result.read(Cassandra.java:15392) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_insert(Cassandra.java:907) [junit] at org.apache.cassandra.thrift.Cassandra$Client.insert(Cassandra.java:879)
[jira] Updated: (CASSANDRA-1964) MutationTest of the distributed-test suite fails
[ https://issues.apache.org/jira/browse/CASSANDRA-1964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-1964: Attachment: 0001-Add-a-builder-for-a-retrying-get-attempt-and-use-to-im.txt MutationTest of the distributed-test suite fails Key: CASSANDRA-1964 URL: https://issues.apache.org/jira/browse/CASSANDRA-1964 Project: Cassandra Issue Type: Bug Components: Core Environment: OS X Version 10.6.6 java version 1.6.0_22 Java(TM) SE Runtime Environment (build 1.6.0_22-b04-307-10M3261) Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03-307, mixed mode) Apache Ant version 1.8.1 compiled on September 21 2010 Reporter: Michael Allen Assignee: Stu Hood Priority: Minor Fix For: 0.8 Attachments: 0001-Add-a-builder-for-a-retrying-get-attempt-and-use-to-im.txt MutationTest of the distributed-test test suite causes errors on trunk. To reproduce, issue: ant distributed-test -Dwhirr.config=path_to_whirr_config_file from the project root. relevant whirr configuration settings used: whirr.service-name=cassandra whirr.cluster-name=cassandra_test whirr.instance-templates=4 cassandra whirr.version=0.3.0-incubating-SNAPSHOT whirr.location-id=us-west-1 whirr.image-id=us-west-1/ami-16f3a253 whirr.hardware-id=m1.large whirr.blobstore.provider=s3 whirr.blobstore.container=tawamuducassandratests whirr.provider=ec2 whirr.run-url-base=http://hoodidge.net/scripts/ Traceback: distributed-test: [echo] running distributed tests [junit] WARNING: multiple versions of ant detected in path for junit [junit] jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class [junit] and jar:file:/Users/mallen/Desktop/cassandra-trunk/build/lib/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class [junit] Testsuite: org.apache.cassandra.MovementTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 446.65 sec [junit] [junit] - Standard Error - [junit] SLF4J: Class path contains multiple SLF4J bindings. [junit] SLF4J: Found binding in [jar:file:/Users/mallen/Desktop/cassandra-trunk/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit] SLF4J: Found binding in [jar:file:/Users/mallen/Desktop/cassandra-trunk/build/test/lib/jars/whirr-cli-0.3.0-incubating-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. [junit] WARN 12:12:46,654 over limit 471283/262144: wrote temp file [junit] WARN 12:12:48,572 over limit 374979/262144: wrote temp file [junit] WARN 12:12:50,701 over limit 892174/262144: wrote temp file [junit] WARN 12:12:54,442 over limit 612358/262144: wrote temp file [junit] - --- [junit] Testsuite: org.apache.cassandra.MutationTest [junit] Tests run: 4, Failures: 0, Errors: 3, Time elapsed: 110.971 sec [junit] [junit] Testcase: testInsert(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] NotFoundException() [junit] at org.apache.cassandra.thrift.Cassandra$get_result.read(Cassandra.java:6900) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:568) [junit] at org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:541) [junit] at org.apache.cassandra.MutationTest.getColumn(MutationTest.java:210) [junit] at org.apache.cassandra.MutationTest.testInsert(MutationTest.java:66) [junit] [junit] [junit] Testcase: testWriteAllReadOne(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] NotFoundException() [junit] at org.apache.cassandra.thrift.Cassandra$get_result.read(Cassandra.java:6900) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:568) [junit] at org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:541) [junit] at org.apache.cassandra.MutationTest.getColumn(MutationTest.java:210) [junit] at org.apache.cassandra.MutationTest.testWriteAllReadOne(MutationTest.java:87) [junit] [junit] [junit] Testcase: testWriteOneReadAll(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] TimedOutException() [junit] at org.apache.cassandra.thrift.Cassandra$insert_result.read(Cassandra.java:15392) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_insert(Cassandra.java:907) [junit] at org.apache.cassandra.thrift.Cassandra$Client.insert(Cassandra.java:879) [junit]
[jira] Commented: (CASSANDRA-1985) read repair on CL.ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981952#action_12981952 ] T Jake Luciani commented on CASSANDRA-1985: --- +1 read repair on CL.ONE regression Key: CASSANDRA-1985 URL: https://issues.apache.org/jira/browse/CASSANDRA-1985 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.1 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1 Attachments: 1985-v2.txt, CASSANDRA-1985-0001-fix-CL.ONE-read-repair-regression.patch read repair w/ CL.ONE had a regression. The RepairCallback was dropped (in the background for CL.ONE), so ReadResponseResolver : resolve() was never called. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1986) write CL ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981960#action_12981960 ] T Jake Luciani commented on CASSANDRA-1986: --- ah good catch, +1 write CL ONE regression - Key: CASSANDRA-1986 URL: https://issues.apache.org/jira/browse/CASSANDRA-1986 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.1 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1 Attachments: CASSANDRA-1986-0001-fix-write-CL-ONE-regression.patch write CL ONE regression by the DC refactor. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CASSANDRA-1987) Make AE repair DC aware
Make AE repair DC aware --- Key: CASSANDRA-1987 URL: https://issues.apache.org/jira/browse/CASSANDRA-1987 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Online replication is now XDC aware, such that writes directed to another DC are sent to one replica in that DC, and are then fanned out. AE repair could be made much more intelligent, and could follow this model as well: only one replica in a replica set should communicate XDC with the replicas in another DC. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1979) CassandraServiceDataCleaner.prepare() fails with IOException.
[ https://issues.apache.org/jira/browse/CASSANDRA-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nate McCall updated CASSANDRA-1979: --- Attachment: 1979.txt This code path had not been tested in a while. Test case had the following issues: - not using bytebuffer - storage-config param from 0.6 - non-framed transport Patch brings everything up to date with current trunk. CassandraServiceDataCleaner.prepare() fails with IOException. - Key: CASSANDRA-1979 URL: https://issues.apache.org/jira/browse/CASSANDRA-1979 Project: Cassandra Issue Type: Bug Components: Contrib Affects Versions: 0.7.0 Environment: Windows XP Reporter: Chris Ng Assignee: Nate McCall Attachments: 1979.txt CassandraServiceDataCleaner.prepare() fails with an IOException if run in isolation. It seems that initializing the DataDescriptor creates a new CommitLog file, and then the cleaner tries to delete this file and fails. 16:06:07.204 [main] INFO o.a.c.config.DatabaseDescriptor - Loading settings from file:/C:/workspace/sandbox/target/classes/cassandra.yaml 16:06:07.282 [main] DEBUG o.a.c.config.DatabaseDescriptor - Syncing log with a period of 1 16:06:07.282 [main] INFO o.a.c.config.DatabaseDescriptor - DiskAccessMode 'auto' determined to be standard, indexAccessMode is standard 16:06:07.797 [main] DEBUG o.a.c.config.DatabaseDescriptor - setting auto_bootstrap to false 16:06:07.797 [main] INFO o.a.c.db.commitlog.CommitLogSegment - Creating new commitlog segment target/var/lib/cassandra/commitlog\CommitLog-1294934767797.log 16:06:07.813 [main] DEBUG o.apache.cassandra.io.util.FileUtils - Deleting CommitLog-1294934767797.log Exception in thread main java.io.IOException: Failed to delete C:\workspace\sandbox\target\var\lib\cassandra\commitlog\CommitLog-1294934767797.log at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:54) at org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:201) at org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner.cleanDir(CassandraServiceDataCleaner.java:99) at org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner.cleanupDataDirectories(CassandraServiceDataCleaner.java:53) at org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner.prepare(CassandraServiceDataCleaner.java:44) at cng.sandbox.App.main(App.java:15) This also seems to leave a bunch of threads running in the background, so the process has to be manually killed. This was tested with the javautils in the 0.7.0 branch. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1979) CassandraServiceDataCleaner.prepare() fails with IOException.
[ https://issues.apache.org/jira/browse/CASSANDRA-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981965#action_12981965 ] Nate McCall commented on CASSANDRA-1979: I'm not sure this belongs in contrib anymore as it will just get out of date again. I would prefer we remove it in favor of doing something smarter in refactoring out o.a.c.SchemaLoader and friends in the test/unit tree to a module of some sort and including that on the test classpath of the main build file. (In conjunction with CASSANDRA-1848 perhaps?) CassandraServiceDataCleaner.prepare() fails with IOException. - Key: CASSANDRA-1979 URL: https://issues.apache.org/jira/browse/CASSANDRA-1979 Project: Cassandra Issue Type: Bug Components: Contrib Affects Versions: 0.7.0 Environment: Windows XP Reporter: Chris Ng Assignee: Nate McCall Attachments: 1979.txt CassandraServiceDataCleaner.prepare() fails with an IOException if run in isolation. It seems that initializing the DataDescriptor creates a new CommitLog file, and then the cleaner tries to delete this file and fails. 16:06:07.204 [main] INFO o.a.c.config.DatabaseDescriptor - Loading settings from file:/C:/workspace/sandbox/target/classes/cassandra.yaml 16:06:07.282 [main] DEBUG o.a.c.config.DatabaseDescriptor - Syncing log with a period of 1 16:06:07.282 [main] INFO o.a.c.config.DatabaseDescriptor - DiskAccessMode 'auto' determined to be standard, indexAccessMode is standard 16:06:07.797 [main] DEBUG o.a.c.config.DatabaseDescriptor - setting auto_bootstrap to false 16:06:07.797 [main] INFO o.a.c.db.commitlog.CommitLogSegment - Creating new commitlog segment target/var/lib/cassandra/commitlog\CommitLog-1294934767797.log 16:06:07.813 [main] DEBUG o.apache.cassandra.io.util.FileUtils - Deleting CommitLog-1294934767797.log Exception in thread main java.io.IOException: Failed to delete C:\workspace\sandbox\target\var\lib\cassandra\commitlog\CommitLog-1294934767797.log at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:54) at org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:201) at org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner.cleanDir(CassandraServiceDataCleaner.java:99) at org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner.cleanupDataDirectories(CassandraServiceDataCleaner.java:53) at org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner.prepare(CassandraServiceDataCleaner.java:44) at cng.sandbox.App.main(App.java:15) This also seems to leave a bunch of threads running in the background, so the process has to be manually killed. This was tested with the javautils in the 0.7.0 branch. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1906) Sanitize configuration code
[ https://issues.apache.org/jira/browse/CASSANDRA-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981978#action_12981978 ] Jon Hermes commented on CASSANDRA-1906: --- - Dropping avro (CASSANDRA-926) makes the [KS,CF,Column]Def changes much easier. I know at least CfDef will be cut in half. - If not dropping YAML entirely, then at least cleaning up the code around it and making a serious effort to use YAMLBeans (previous reverted yaml lib changes (CASSANDRA-1313) removed 100 lines of code, and YAMLBeans itself is ~10% the mass of snakeYAML) should be done. The differences between the libraries are now known. - Taking a census of all the settings knobs and sorting them into JMX vs. Thrift needs to be done to make the separation consistent, or if this isn't done, then we should have all methods available on both Thrift and JMX. In either case, the reasoning is that future methods won't have to endure an argument as to their placement; i.e. either it obviously goes in one, or necessarily goes in both. The core tenet of this improvement/ticket is making the code easier for future maintenance; the more unnecessary lines removed and the more commentary and documentation available the better. There's currently a *ton* of redundant code and deprecated flags running around in DD, the Defs, and SS, and missing a single spot of redundancy means an unusable feature (CASSANDRA-1606, CASSANDRA-1466). Sanitize configuration code --- Key: CASSANDRA-1906 URL: https://issues.apache.org/jira/browse/CASSANDRA-1906 Project: Cassandra Issue Type: Improvement Reporter: Jon Hermes Assignee: Jon Hermes Priority: Minor Fix For: 0.8 Original Estimate: 24h Remaining Estimate: 24h Multipart: - Drop deprecated YAML config. Only config allowed is via thrift/JMX. Make this gratuitously easy to do with sane defaults and accepting changesets as opposed to full definitions. - Combine common code between KS/CF/ColumnDefs and between thrift/avro defs. - Provide an obvious and clean interface for changing settings locally versus globally (JMX vs. thrift). Dox here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-1986) write CL ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani resolved CASSANDRA-1986. --- Resolution: Fixed Reviewer: tjake write CL ONE regression - Key: CASSANDRA-1986 URL: https://issues.apache.org/jira/browse/CASSANDRA-1986 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.1 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1 Attachments: CASSANDRA-1986-0001-fix-write-CL-ONE-regression.patch write CL ONE regression by the DC refactor. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1986) write CL ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12982011#action_12982011 ] Hudson commented on CASSANDRA-1986: --- Integrated in Cassandra-0.7 #160 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/160/]) write CL ONE regression - Key: CASSANDRA-1986 URL: https://issues.apache.org/jira/browse/CASSANDRA-1986 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.1 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1 Attachments: CASSANDRA-1986-0001-fix-write-CL-ONE-regression.patch write CL ONE regression by the DC refactor. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12982009#action_12982009 ] Hudson commented on CASSANDRA-1714: --- Integrated in Cassandra-0.7 #160 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/160/]) zero-copy reads Patch by Pavel Yaskevich; Reviewed by Jake Luciani for CASSANDRA-1714 zero-copy reads --- Key: CASSANDRA-1714 URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: 1714-v5-07.txt, CASSANDRA-1714-0.7.patch, CASSANDRA-1714-v2.patch, CASSANDRA-1714-v3.patch, CASSANDRA-1714-v4-trunk.patch, CASSANDRA-1714-v6-0.7.patch, CASSANDRA-1714.patch, zerocopy.txt Original Estimate: 32h Time Spent: 32h Remaining Estimate: 0h Since we are already using mmap'd ByteBuffers in MappedFileDataInput we should be able to do zero-copy reads (via buffer.slice()), which would give us better performance than CASSANDRA-1651 without having to worry about buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1530) More-efficient cross-DC replication
[ https://issues.apache.org/jira/browse/CASSANDRA-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12982010#action_12982010 ] Hudson commented on CASSANDRA-1530: --- Integrated in Cassandra-0.7 #160 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/160/]) More-efficient cross-DC replication --- Key: CASSANDRA-1530 URL: https://issues.apache.org/jira/browse/CASSANDRA-1530 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: T Jake Luciani Fix For: 0.7.1 Attachments: 1530-cleaned.txt, 1530-joaquin.txt, 1530-v5.txt, 1530_v2.txt Original Estimate: 24h Remaining Estimate: 24h Currently the coordinator node sends writes to each replica. In multi-dc situations, it would be more efficient to have the coordinator talk to a single node in each DC, which would forward the writes to the other replicas local to it. (All replicas would continue to ack to the coordinator.) Shouldn't be difficult to add a header to RMVH saying consult the Strategy to see who else is local to you and queue it off to them as well. For example, consider two data centers with the following nodes: DC1: A, B, C DC2: D, E, F A client connected to node A performs an insert that should be replicated to {B,C,E,F}. Currently, A will send a MUTATION verb to each of those nodes directly. But communication to E and F is relatively expensive since they are in the other data center from A. So we'd like to send the mutation to just one, say E, with a tag saying forward this to F as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1964) MutationTest of the distributed-test suite fails
[ https://issues.apache.org/jira/browse/CASSANDRA-1964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-1964: Attachment: (was: 0001-Add-a-builder-for-a-retrying-get-attempt-and-use-to-im.txt) MutationTest of the distributed-test suite fails Key: CASSANDRA-1964 URL: https://issues.apache.org/jira/browse/CASSANDRA-1964 Project: Cassandra Issue Type: Bug Components: Core Environment: OS X Version 10.6.6 java version 1.6.0_22 Java(TM) SE Runtime Environment (build 1.6.0_22-b04-307-10M3261) Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03-307, mixed mode) Apache Ant version 1.8.1 compiled on September 21 2010 Reporter: Michael Allen Assignee: Stu Hood Priority: Minor Fix For: 0.8 MutationTest of the distributed-test test suite causes errors on trunk. To reproduce, issue: ant distributed-test -Dwhirr.config=path_to_whirr_config_file from the project root. relevant whirr configuration settings used: whirr.service-name=cassandra whirr.cluster-name=cassandra_test whirr.instance-templates=4 cassandra whirr.version=0.3.0-incubating-SNAPSHOT whirr.location-id=us-west-1 whirr.image-id=us-west-1/ami-16f3a253 whirr.hardware-id=m1.large whirr.blobstore.provider=s3 whirr.blobstore.container=tawamuducassandratests whirr.provider=ec2 whirr.run-url-base=http://hoodidge.net/scripts/ Traceback: distributed-test: [echo] running distributed tests [junit] WARNING: multiple versions of ant detected in path for junit [junit] jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class [junit] and jar:file:/Users/mallen/Desktop/cassandra-trunk/build/lib/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class [junit] Testsuite: org.apache.cassandra.MovementTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 446.65 sec [junit] [junit] - Standard Error - [junit] SLF4J: Class path contains multiple SLF4J bindings. [junit] SLF4J: Found binding in [jar:file:/Users/mallen/Desktop/cassandra-trunk/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit] SLF4J: Found binding in [jar:file:/Users/mallen/Desktop/cassandra-trunk/build/test/lib/jars/whirr-cli-0.3.0-incubating-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. [junit] WARN 12:12:46,654 over limit 471283/262144: wrote temp file [junit] WARN 12:12:48,572 over limit 374979/262144: wrote temp file [junit] WARN 12:12:50,701 over limit 892174/262144: wrote temp file [junit] WARN 12:12:54,442 over limit 612358/262144: wrote temp file [junit] - --- [junit] Testsuite: org.apache.cassandra.MutationTest [junit] Tests run: 4, Failures: 0, Errors: 3, Time elapsed: 110.971 sec [junit] [junit] Testcase: testInsert(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] NotFoundException() [junit] at org.apache.cassandra.thrift.Cassandra$get_result.read(Cassandra.java:6900) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:568) [junit] at org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:541) [junit] at org.apache.cassandra.MutationTest.getColumn(MutationTest.java:210) [junit] at org.apache.cassandra.MutationTest.testInsert(MutationTest.java:66) [junit] [junit] [junit] Testcase: testWriteAllReadOne(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] NotFoundException() [junit] at org.apache.cassandra.thrift.Cassandra$get_result.read(Cassandra.java:6900) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:568) [junit] at org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:541) [junit] at org.apache.cassandra.MutationTest.getColumn(MutationTest.java:210) [junit] at org.apache.cassandra.MutationTest.testWriteAllReadOne(MutationTest.java:87) [junit] [junit] [junit] Testcase: testWriteOneReadAll(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] TimedOutException() [junit] at org.apache.cassandra.thrift.Cassandra$insert_result.read(Cassandra.java:15392) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_insert(Cassandra.java:907) [junit] at org.apache.cassandra.thrift.Cassandra$Client.insert(Cassandra.java:879) [junit] at org.apache.cassandra.MutationTest.insert(MutationTest.java:202)
[jira] Created: (CASSANDRA-1988) Prefer to throw Unavailable rather than Timeout
Prefer to throw Unavailable rather than Timeout --- Key: CASSANDRA-1988 URL: https://issues.apache.org/jira/browse/CASSANDRA-1988 Project: Cassandra Issue Type: Improvement Components: API Reporter: Stu Hood Fix For: 0.8 When a node is unreachable, but is not yet being reported dead by gossip, messages are enqueued in the messaging service to be sent when the node becomes available again (on the assumption that the connection dropped temporarily). Higher up in the client layer, before sending messages to other nodes, we check that they are alive according to gossip, and fail fast with UnavailableException if they are not (CASSANDRA-1803). If we send messages to nodes that are not yet being reported dead, the messages sit in queue, and time out rather than being sent: this results in the client request failing with a TimeoutException. If we differentiate between messages that were never sent (aka, are still queued in the MessagingService at the end of the timeout), and messages that were sent but didn't get a response, we can properly throw UnavailableException in the former case. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1964) MutationTest of the distributed-test suite fails
[ https://issues.apache.org/jira/browse/CASSANDRA-1964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-1964: Attachment: 0001-Add-a-builder-for-a-retrying-get-attempt-and-use-to-im.txt Ready for review: CASSANDRA-1986 is closed, and we pass all tests consistently. I was a little bit uncomfortable having to set such high retry timeouts in cases where we need to wait for gossip (in particular when waiting for the fail fast behaviour from CASSANDRA-1803 to kick in), but it is a bit of a necessary evil with cloud providers. I opened CASSANDRA-1988 to discuss ways to improve this behaviour. MutationTest of the distributed-test suite fails Key: CASSANDRA-1964 URL: https://issues.apache.org/jira/browse/CASSANDRA-1964 Project: Cassandra Issue Type: Bug Components: Core Environment: OS X Version 10.6.6 java version 1.6.0_22 Java(TM) SE Runtime Environment (build 1.6.0_22-b04-307-10M3261) Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03-307, mixed mode) Apache Ant version 1.8.1 compiled on September 21 2010 Reporter: Michael Allen Assignee: Stu Hood Priority: Minor Fix For: 0.8 Attachments: 0001-Add-a-builder-for-a-retrying-get-attempt-and-use-to-im.txt MutationTest of the distributed-test test suite causes errors on trunk. To reproduce, issue: ant distributed-test -Dwhirr.config=path_to_whirr_config_file from the project root. relevant whirr configuration settings used: whirr.service-name=cassandra whirr.cluster-name=cassandra_test whirr.instance-templates=4 cassandra whirr.version=0.3.0-incubating-SNAPSHOT whirr.location-id=us-west-1 whirr.image-id=us-west-1/ami-16f3a253 whirr.hardware-id=m1.large whirr.blobstore.provider=s3 whirr.blobstore.container=tawamuducassandratests whirr.provider=ec2 whirr.run-url-base=http://hoodidge.net/scripts/ Traceback: distributed-test: [echo] running distributed tests [junit] WARNING: multiple versions of ant detected in path for junit [junit] jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class [junit] and jar:file:/Users/mallen/Desktop/cassandra-trunk/build/lib/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class [junit] Testsuite: org.apache.cassandra.MovementTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 446.65 sec [junit] [junit] - Standard Error - [junit] SLF4J: Class path contains multiple SLF4J bindings. [junit] SLF4J: Found binding in [jar:file:/Users/mallen/Desktop/cassandra-trunk/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit] SLF4J: Found binding in [jar:file:/Users/mallen/Desktop/cassandra-trunk/build/test/lib/jars/whirr-cli-0.3.0-incubating-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. [junit] WARN 12:12:46,654 over limit 471283/262144: wrote temp file [junit] WARN 12:12:48,572 over limit 374979/262144: wrote temp file [junit] WARN 12:12:50,701 over limit 892174/262144: wrote temp file [junit] WARN 12:12:54,442 over limit 612358/262144: wrote temp file [junit] - --- [junit] Testsuite: org.apache.cassandra.MutationTest [junit] Tests run: 4, Failures: 0, Errors: 3, Time elapsed: 110.971 sec [junit] [junit] Testcase: testInsert(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] NotFoundException() [junit] at org.apache.cassandra.thrift.Cassandra$get_result.read(Cassandra.java:6900) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:568) [junit] at org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:541) [junit] at org.apache.cassandra.MutationTest.getColumn(MutationTest.java:210) [junit] at org.apache.cassandra.MutationTest.testInsert(MutationTest.java:66) [junit] [junit] [junit] Testcase: testWriteAllReadOne(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] NotFoundException() [junit] at org.apache.cassandra.thrift.Cassandra$get_result.read(Cassandra.java:6900) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:568) [junit] at org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:541) [junit] at org.apache.cassandra.MutationTest.getColumn(MutationTest.java:210) [junit] at org.apache.cassandra.MutationTest.testWriteAllReadOne(MutationTest.java:87) [junit] [junit] [junit] Testcase:
[jira] Commented: (CASSANDRA-926) remove alternative RPC interface using Avro
[ https://issues.apache.org/jira/browse/CASSANDRA-926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12982022#action_12982022 ] Jeremy Hanna commented on CASSANDRA-926: Looks good. Two little things. - In contrib/word_count/src/WordCount.java, the package for the avro stuff needs to change from org.apache.cassandra.avro.XXX to org.apache.cassandra.hadoop.avro.XXX. - In contrib/hadoop_output_streaming, the line 35 needs to change to: -file $cwd/../../../src/gen-java/org/apache/cassandra/hadoop/hadoop.avpr \ Other than that, looks good to me. remove alternative RPC interface using Avro --- Key: CASSANDRA-926 URL: https://issues.apache.org/jira/browse/CASSANDRA-926 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Eric Evans Assignee: Eric Evans Priority: Minor Attachments: v1-0001-CASSANDRA-926-denormalize-avro-internode-rpc-schema.txt, v1-0002-don-t-build-schema-for-rpc-interface.txt, v1-0003-remove-option-to-launch-avro-rpc-daemon.txt, v1-0004-remove-avro-functional-tests.txt, v1-0005-remove-avro-rpc-source.txt, v1-0006-move-remaining-avro-records-to-hadoop-package.txt, v2-0001-CASSANDRA-926-denormalize-avro-internode-rpc-schema.txt, v2-0002-don-t-build-schema-for-rpc-interface.txt, v2-0003-remove-option-to-launch-avro-rpc-daemon.txt, v2-0004-remove-avro-functional-tests.txt, v2-0005-remove-avro-rpc-source.txt, v2-0006-move-remaining-avro-records-to-hadoop-package.txt Original Estimate: 0h Remaining Estimate: 0h *UPDATE: This ticket is now for the removal of the Avro interface, (see http://thread.gmane.org/gmane.comp.db.cassandra.client.devel/36 for background).* Avro is data serialization and RPC framework similar to Thrift. It provides: * Rich data structures. * A compact, fast, binary data format. * A container file, to store persistent data. * Remote procedure call (RPC). * Simple integration with dynamic languages. Code generation is not required to read or write data files nor to use or implement RPC protocols. Code generation as an optional optimization, only worth implementing for statically typed languages. Cassandra's Avro interface is being structured in a way that closely mirrors the existing Thrift interface, both in terms of public facing API, and how it is implemented. GSOC students interested in this task should begin by familiarizing themselves with Cassandra's Thrift service (org.apache.cassandra.thrift). Note: This is a very large and long-running task so treat this as a meta-issue and add sub-tasks and/or blocking issues as appropriate. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-926) remove alternative RPC interface using Avro
[ https://issues.apache.org/jira/browse/CASSANDRA-926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12982022#action_12982022 ] Jeremy Hanna edited comment on CASSANDRA-926 at 1/14/11 11:52 PM: -- Looks good. Two little things. - In contrib/word_count/src/WordCount.java, the package for the avro stuff needs to change from org.apache.cassandra.avro.XXX to org.apache.cassandra.hadoop.avro.XXX. - In contrib/hadoop_output_streaming/bin/streaming, line 35 needs to change to: -file $cwd/../../../src/gen-java/org/apache/cassandra/hadoop/hadoop.avpr \ Other than that, looks good to me. was (Author: jeromatron): Looks good. Two little things. - In contrib/word_count/src/WordCount.java, the package for the avro stuff needs to change from org.apache.cassandra.avro.XXX to org.apache.cassandra.hadoop.avro.XXX. - In contrib/hadoop_output_streaming, the line 35 needs to change to: -file $cwd/../../../src/gen-java/org/apache/cassandra/hadoop/hadoop.avpr \ Other than that, looks good to me. remove alternative RPC interface using Avro --- Key: CASSANDRA-926 URL: https://issues.apache.org/jira/browse/CASSANDRA-926 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Eric Evans Assignee: Eric Evans Priority: Minor Attachments: v1-0001-CASSANDRA-926-denormalize-avro-internode-rpc-schema.txt, v1-0002-don-t-build-schema-for-rpc-interface.txt, v1-0003-remove-option-to-launch-avro-rpc-daemon.txt, v1-0004-remove-avro-functional-tests.txt, v1-0005-remove-avro-rpc-source.txt, v1-0006-move-remaining-avro-records-to-hadoop-package.txt, v2-0001-CASSANDRA-926-denormalize-avro-internode-rpc-schema.txt, v2-0002-don-t-build-schema-for-rpc-interface.txt, v2-0003-remove-option-to-launch-avro-rpc-daemon.txt, v2-0004-remove-avro-functional-tests.txt, v2-0005-remove-avro-rpc-source.txt, v2-0006-move-remaining-avro-records-to-hadoop-package.txt Original Estimate: 0h Remaining Estimate: 0h *UPDATE: This ticket is now for the removal of the Avro interface, (see http://thread.gmane.org/gmane.comp.db.cassandra.client.devel/36 for background).* Avro is data serialization and RPC framework similar to Thrift. It provides: * Rich data structures. * A compact, fast, binary data format. * A container file, to store persistent data. * Remote procedure call (RPC). * Simple integration with dynamic languages. Code generation is not required to read or write data files nor to use or implement RPC protocols. Code generation as an optional optimization, only worth implementing for statically typed languages. Cassandra's Avro interface is being structured in a way that closely mirrors the existing Thrift interface, both in terms of public facing API, and how it is implemented. GSOC students interested in this task should begin by familiarizing themselves with Cassandra's Thrift service (org.apache.cassandra.thrift). Note: This is a very large and long-running task so treat this as a meta-issue and add sub-tasks and/or blocking issues as appropriate. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1988) Prefer to throw Unavailable rather than Timeout
[ https://issues.apache.org/jira/browse/CASSANDRA-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12982023#action_12982023 ] Stu Hood commented on CASSANDRA-1988: - In particular, on CASSANDRA-1964, the lag between the connection dying and gossip noticing the death was up to 20 seconds, meaning we received TimeoutException twice before getting an UnavailableException. Prefer to throw Unavailable rather than Timeout --- Key: CASSANDRA-1988 URL: https://issues.apache.org/jira/browse/CASSANDRA-1988 Project: Cassandra Issue Type: Improvement Components: API Reporter: Stu Hood Fix For: 0.8 When a node is unreachable, but is not yet being reported dead by gossip, messages are enqueued in the messaging service to be sent when the node becomes available again (on the assumption that the connection dropped temporarily). Higher up in the client layer, before sending messages to other nodes, we check that they are alive according to gossip, and fail fast with UnavailableException if they are not (CASSANDRA-1803). If we send messages to nodes that are not yet being reported dead, the messages sit in queue, and time out rather than being sent: this results in the client request failing with a TimeoutException. If we differentiate between messages that were never sent (aka, are still queued in the MessagingService at the end of the timeout), and messages that were sent but didn't get a response, we can properly throw UnavailableException in the former case. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1944) Support CL.QUORUM and CL.ALL for counter writes
[ https://issues.apache.org/jira/browse/CASSANDRA-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12982024#action_12982024 ] Jonathan Ellis commented on CASSANDRA-1944: --- Is this ready to commit? Support CL.QUORUM and CL.ALL for counter writes Key: CASSANDRA-1944 URL: https://issues.apache.org/jira/browse/CASSANDRA-1944 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 0.8 Attachments: 0001-Add-support-for-CL.QUORUM-and-CL.ALL.patch Original Estimate: 16h Remaining Estimate: 16h This ticket basically proposes to make the counter ReplicateOnWrite mechanism synchronous instead of asynchronous. This will allow to support CL.QUORUM and CL.ALL for counter writes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1923) unit tests that validate that message serialization isn't broken in the current version.
[ https://issues.apache.org/jira/browse/CASSANDRA-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12982025#action_12982025 ] Jonathan Ellis commented on CASSANDRA-1923: --- I take it a new version of the patches is planned? unit tests that validate that message serialization isn't broken in the current version. Key: CASSANDRA-1923 URL: https://issues.apache.org/jira/browse/CASSANDRA-1923 Project: Cassandra Issue Type: Sub-task Components: Core Reporter: Gary Dusbabek Assignee: Gary Dusbabek Priority: Minor Fix For: 0.7.1 There are two components to this. First, code that will generate the serialized messages. Second, code that will attempt to read the serialized messages. My plan is to commit this to 0.7.1 with generated serialized messages. Then I will merge that into trunk sans the generation code. A similar process will need to take place when we branch trunk to create 0.8, etc. On second thought, maybe it makes sense to keep the generation code and let it morph as the message formats change. If the tests ever break in the 0.7 branch, that means we've created a message incompatibility regression that needs to be fixed. If the tests ever break in trunk (post CASSANDRA-1015), it means that something in trunk has changed message serialization compatibility that will need to be restored (via whatever process is used for 1015). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1895) Loadbalance during gossip issues leaves cluster in bad state
[ https://issues.apache.org/jira/browse/CASSANDRA-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12982026#action_12982026 ] Jonathan Ellis commented on CASSANDRA-1895: --- would it make more sense to change FatClientTimeout to QUARANTINE_DELAY / 2? Loadbalance during gossip issues leaves cluster in bad state Key: CASSANDRA-1895 URL: https://issues.apache.org/jira/browse/CASSANDRA-1895 Project: Cassandra Issue Type: Bug Components: Core Reporter: Stu Hood Assignee: Brandon Williams Priority: Minor Fix For: 0.7.1 Attachments: 1895.txt, logs.tgz, ring-views.txt Original Estimate: 8h Remaining Estimate: 8h Running loadbalance against a node in a 4 node cluster leaves gossip in a wonky state. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CASSANDRA-1989) Cannot parse generation after restart
Cannot parse generation after restart - Key: CASSANDRA-1989 URL: https://issues.apache.org/jira/browse/CASSANDRA-1989 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 Reporter: Stu Hood Assignee: T Jake Luciani Fix For: 0.8 Looks like CASSANDRA-1714 broke some parsing of the generation on some restarts: haven't tracked it down yet, but this is likely to be obvious for someone who worked on that issue. {code}java.lang.UnsupportedOperationException at java.nio.ByteBuffer.array(ByteBuffer.java:940) at org.apache.cassandra.utils.FBUtilities.byteBufferToInt(FBUtilities.java:212) at org.apache.cassandra.db.SystemTable.incrementAndGetGeneration(SystemTable.java:286) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:356) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:184) at org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:54) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:240) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:133){code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1989) Cannot parse generation after restart
[ https://issues.apache.org/jira/browse/CASSANDRA-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1989: -- Affects Version/s: (was: 0.8) 0.7.1 Fix Version/s: (was: 0.8) 0.7.1 Assignee: Pavel Yaskevich (was: T Jake Luciani) Cannot parse generation after restart - Key: CASSANDRA-1989 URL: https://issues.apache.org/jira/browse/CASSANDRA-1989 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.1 Reporter: Stu Hood Assignee: Pavel Yaskevich Fix For: 0.7.1 Looks like CASSANDRA-1714 broke some parsing of the generation on some restarts: haven't tracked it down yet, but this is likely to be obvious for someone who worked on that issue. {code}java.lang.UnsupportedOperationException at java.nio.ByteBuffer.array(ByteBuffer.java:940) at org.apache.cassandra.utils.FBUtilities.byteBufferToInt(FBUtilities.java:212) at org.apache.cassandra.db.SystemTable.incrementAndGetGeneration(SystemTable.java:286) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:356) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:184) at org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:54) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:240) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:133){code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12982032#action_12982032 ] Jonathan Ellis commented on CASSANDRA-1714: --- I think we need a trunk version of the v6 patch. zero-copy reads --- Key: CASSANDRA-1714 URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: 1714-v5-07.txt, CASSANDRA-1714-0.7.patch, CASSANDRA-1714-v2.patch, CASSANDRA-1714-v3.patch, CASSANDRA-1714-v4-trunk.patch, CASSANDRA-1714-v6-0.7.patch, CASSANDRA-1714.patch, zerocopy.txt Original Estimate: 32h Time Spent: 32h Remaining Estimate: 0h Since we are already using mmap'd ByteBuffers in MappedFileDataInput we should be able to do zero-copy reads (via buffer.slice()), which would give us better performance than CASSANDRA-1651 without having to worry about buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.