[jira] [Commented] (CASSANDRA-2735) Timestamp Based Compaction Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108467#comment-13108467 ] Viktor Jevdokimov commented on CASSANDRA-2735: -- When this patch will be included in any public release? Timestamp Based Compaction Strategy --- Key: CASSANDRA-2735 URL: https://issues.apache.org/jira/browse/CASSANDRA-2735 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Alan Liang Assignee: Alan Liang Priority: Minor Labels: compaction Attachments: 0001-timestamp-bucketed-compaction-strategy-V2.patch, 0001-timestamp-bucketed-compaction-strategy.patch Compaction strategy implementation based on max timestamp ordering of the sstables while satisfying max sstable size, min and max compaction thresholds. It also handles expiration of sstables based on a timestamp. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1173049 - /cassandra/tags/cassandra-0.8.6/
Author: slebresne Date: Tue Sep 20 09:35:34 2011 New Revision: 1173049 URL: http://svn.apache.org/viewvc?rev=1173049view=rev Log: Create tag for 0.8.6 release Added: cassandra/tags/cassandra-0.8.6/ (props changed) - copied from r1171107, cassandra/branches/cassandra-0.8/ Propchange: cassandra/tags/cassandra-0.8.6/ -- --- svn:ignore (added) +++ svn:ignore Tue Sep 20 09:35:34 2011 @@ -0,0 +1,8 @@ +.classpath +.project +.settings +temp-testng-customsuite.xml +build +build.properties +.idea +out Propchange: cassandra/tags/cassandra-0.8.6/ -- --- svn:mergeinfo (added) +++ svn:mergeinfo Tue Sep 20 09:35:34 2011 @@ -0,0 +1,12 @@ +/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 +/cassandra/branches/cassandra-0.7:1026516-1170333 +/cassandra/branches/cassandra-0.7.0:1053690-1055654 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125041 +/cassandra/branches/cassandra-0.8.0:1125021-1130369 +/cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 +/cassandra/tags/cassandra-0.8.0-rc1:1102511-1125020 +/cassandra/trunk:1129049-1129050,1129065,1151625,1152416,1153189 +/incubator/cassandra/branches/cassandra-0.3:774578-796573 +/incubator/cassandra/branches/cassandra-0.4:810145-834239,834349-834350 +/incubator/cassandra/branches/cassandra-0.5:72-915439 +/incubator/cassandra/branches/cassandra-0.6:911237-922688
[jira] [Created] (CASSANDRA-3232) Startup after abrupt shutdown may result in failure to delete a temporary SSTable
Startup after abrupt shutdown may result in failure to delete a temporary
SSTable
-
Key: CASSANDRA-3232
URL: https://issues.apache.org/jira/browse/CASSANDRA-3232
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.5
Environment: Windows 7 64bit
Reporter: Ophir Radnitz
Priority: Minor
After killing the server and starting it up again, once in 3-4 times we're
encountering the described problem.
A relevant stack trace:
{{
11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see
http://wiki.apache.org/cassandra/MemtableSSTable)
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[cassandra_server,5,main]
java.io.IOError: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
at
org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
at
org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
at
com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
... 6 more
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[FlushWriter:1,5,main]
java.io.IOError: java.io.IOException: rename failed of
c:\cassandra\data\system\LocationInfo-g-24-Index.db
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: rename failed of
c:\cassandra\data\system\LocationInfo-g-24-Index.db
at
org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
... 9 more
11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
11:10:44 SSTableReader [INFO] Opening
c:\cassandra\data\20110914\session_hits-g-456
...
}}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3232) Startup after abrupt shutdown may result in failure to delete a temporary SSTable
[
https://issues.apache.org/jira/browse/CASSANDRA-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ophir Radnitz updated CASSANDRA-3232:
-
Description:
After killing the server and starting it up again, once in 3-4 times we're
encountering the described problem.
A relevant stack trace:
{noformat}
11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see
http://wiki.apache.org/cassandra/MemtableSSTable)
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[cassandra_server,5,main]
java.io.IOError: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
at
org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
at
org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
at
com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
... 6 more
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[FlushWriter:1,5,main]
java.io.IOError: java.io.IOException: rename failed of
c:\cassandra\data\system\LocationInfo-g-24-Index.db
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: rename failed of
c:\cassandra\data\system\LocationInfo-g-24-Index.db
at
org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
... 9 more
11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
11:10:44 SSTableReader [INFO] Opening
c:\cassandra\data\20110914\session_hits-g-456
...
{noformat}
was:
After killing the server and starting it up again, once in 3-4 times we're
encountering the described problem.
A relevant stack trace:
{{
11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see
http://wiki.apache.org/cassandra/MemtableSSTable)
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[cassandra_server,5,main]
java.io.IOError: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
at
org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
at
org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
at
com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
... 6 more
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[FlushWriter:1,5,main]
java.io.IOError: java.io.IOException: rename failed of
[jira] [Created] (CASSANDRA-3233) fatar exception after decommission
fatar exception after decommission -- Key: CASSANDRA-3233 URL: https://issues.apache.org/jira/browse/CASSANDRA-3233 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.5 Environment: linux 2.6.32 (openvz, debian) java sun 1.6.0_26: sun-java6-bin 6.26-0squeeze1 sun-java6-jre 6.26-0squeeze1 Reporter: Zenek Kraweznik after decommission node (succesful) i found in logfile information about fatal exception: INFO 20:19:16,753 Streaming to /10.117.199.233 INFO 21:11:11,699 Removing token 127605887595351923798765477786913079296 for /10.117.199.234 INFO 21:11:11,699 Deleting any stored hints for 10.117.199.234 INFO 21:11:11,701 Announcing that I have left the ring for 3ms INFO 21:11:11,702 Enqueuing flush of Memtable-HintsColumnFamily@1635064453(0/0 serialized/live bytes, 1 ops) INFO 21:11:11,703 Writing Memtable-HintsColumnFamily@1635064453(0/0 serialized/live bytes, 1 ops) INFO 21:11:11,729 Completed flushing /var/lib/cassandra/data/system/HintsColumnFamily-g-1-Data.db (64 bytes) INFO 21:11:41,702 Shutting down MessageService... INFO 21:11:41,704 Shutdown complete (no further commands will be processed) INFO 21:11:41,704 MessagingService shutting down server thread. INFO 21:11:41,706 Decommissioned ERROR 21:11:44,261 Fatal exception in thread Thread[Thread-9632,5,main] java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut down at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:60) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658) at org.apache.cassandra.net.MessagingService.receive(MessagingService.java:490) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:133) INFO 21:43:22,198 Cassandra shutting down... INFO 21:43:22,199 Stop listening to thrift clients 2 last lines are about stopping cassandra via init.d script. Is that exception normal? If it is, then it should be not exception but normal operation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1173061 - in /cassandra/site: publish/download/index.html publish/index.html src/settings.py
Author: slebresne
Date: Tue Sep 20 10:10:55 2011
New Revision: 1173061
URL: http://svn.apache.org/viewvc?rev=1173061view=rev
Log:
Update website for 0.8.6 release
Modified:
cassandra/site/publish/download/index.html
cassandra/site/publish/index.html
cassandra/site/src/settings.py
Modified: cassandra/site/publish/download/index.html
URL:
http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1173061r1=1173060r2=1173061view=diff
==
--- cassandra/site/publish/download/index.html (original)
+++ cassandra/site/publish/download/index.html Tue Sep 20 10:10:55 2011
@@ -73,31 +73,31 @@
p
- The latest stable release of Apache Cassandra is 0.8.5
- (released on 2011-09-08). iIf you're just
+ The latest stable release of Apache Cassandra is 0.8.6
+ (released on 2011-09-20). iIf you're just
starting out, download this one./i
/p
ul
li
a class=filename
-
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.8.5/apache-cassandra-0.8.5-bin.tar.gz;
+
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.8.6/apache-cassandra-0.8.6-bin.tar.gz;
onclick=javascript:
pageTracker._trackPageview('/clicks/binary_download');
- apache-cassandra-0.8.5-bin.tar.gz
+ apache-cassandra-0.8.6-bin.tar.gz
/a
-[a
href=http://www.apache.org/dist/cassandra/0.8.5/apache-cassandra-0.8.5-bin.tar.gz.asc;PGP/a]
-[a
href=http://www.apache.org/dist/cassandra/0.8.5/apache-cassandra-0.8.5-bin.tar.gz.md5;MD5/a]
-[a
href=http://www.apache.org/dist/cassandra/0.8.5/apache-cassandra-0.8.5-bin.tar.gz.sha;SHA1/a]
+[a
href=http://www.apache.org/dist/cassandra/0.8.6/apache-cassandra-0.8.6-bin.tar.gz.asc;PGP/a]
+[a
href=http://www.apache.org/dist/cassandra/0.8.6/apache-cassandra-0.8.6-bin.tar.gz.md5;MD5/a]
+[a
href=http://www.apache.org/dist/cassandra/0.8.6/apache-cassandra-0.8.6-bin.tar.gz.sha;SHA1/a]
/li
li
a class=filename
-
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.8.5/apache-cassandra-0.8.5-src.tar.gz;
+
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.8.6/apache-cassandra-0.8.6-src.tar.gz;
onclick=javascript:
pageTracker._trackPageview('/clicks/source_download');
- apache-cassandra-0.8.5-src.tar.gz
+ apache-cassandra-0.8.6-src.tar.gz
/a
-[a
href=http://www.apache.org/dist/cassandra/0.8.5/apache-cassandra-0.8.5-src.tar.gz.asc;PGP/a]
-[a
href=http://www.apache.org/dist/cassandra/0.8.5/apache-cassandra-0.8.5-src.tar.gz.md5;MD5/a]
-[a
href=http://www.apache.org/dist/cassandra/0.8.5/apache-cassandra-0.8.5-src.tar.gz.sha;SHA1/a]
+[a
href=http://www.apache.org/dist/cassandra/0.8.6/apache-cassandra-0.8.6-src.tar.gz.asc;PGP/a]
+[a
href=http://www.apache.org/dist/cassandra/0.8.6/apache-cassandra-0.8.6-src.tar.gz.md5;MD5/a]
+[a
href=http://www.apache.org/dist/cassandra/0.8.6/apache-cassandra-0.8.6-src.tar.gz.sha;SHA1/a]
/li
/ul
@@ -182,15 +182,15 @@ New users to Cassandra should be sure to
h2Download/h2
div class=inner rc
p
-The latest release is b0.8.5/b
-span class=relnotes(a
href=https://svn.apache.org/repos/asf/cassandra/tags/cassandra-0.8.5/CHANGES.txt;Changes/a)/span
+The latest release is b0.8.6/b
+span class=relnotes(a
href=https://svn.apache.org/repos/asf/cassandra/tags/cassandra-0.8.6/CHANGES.txt;Changes/a)/span
/p
p
a class=filename
-
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.8.5/apache-cassandra-0.8.5-bin.tar.gz;
+
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.8.6/apache-cassandra-0.8.6-bin.tar.gz;
onclick=javascript:
pageTracker._trackPageview('/clicks/binary_download');
- apache-cassandra-0.8.5-bin.tar.gz
+ apache-cassandra-0.8.6-bin.tar.gz
/a
/p
Modified: cassandra/site/publish/index.html
URL:
http://svn.apache.org/viewvc/cassandra/site/publish/index.html?rev=1173061r1=1173060r2=1173061view=diff
==
--- cassandra/site/publish/index.html (original)
+++ cassandra/site/publish/index.html Tue Sep 20 10:10:55 2011
@@ -82,15 +82,15 @@
h2Download/h2
div class=inner rc
p
-The latest release is b0.8.5/b
-span class=relnotes(a
href=https://svn.apache.org/repos/asf/cassandra/tags/cassandra-0.8.5/CHANGES.txt;Changes/a)/span
+The latest release is b0.8.6/b
+span class=relnotes(a
href=https://svn.apache.org/repos/asf/cassandra/tags/cassandra-0.8.6/CHANGES.txt;Changes/a)/span
/p
p
a class=filename
-
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.8.5/apache-cassandra-0.8.5-bin.tar.gz;
+
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.8.6/apache-cassandra-0.8.6-bin.tar.gz;
onclick=javascript:
[jira] [Resolved] (CASSANDRA-3233) fatar exception after decommission
[ https://issues.apache.org/jira/browse/CASSANDRA-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-3233. --- Resolution: Not A Problem This is harmless. fatar exception after decommission -- Key: CASSANDRA-3233 URL: https://issues.apache.org/jira/browse/CASSANDRA-3233 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.5 Environment: linux 2.6.32 (openvz, debian) java sun 1.6.0_26: sun-java6-bin 6.26-0squeeze1 sun-java6-jre 6.26-0squeeze1 Reporter: Zenek Kraweznik after decommission node (succesful) i found in logfile information about fatal exception: INFO 20:19:16,753 Streaming to /10.117.199.233 INFO 21:11:11,699 Removing token 127605887595351923798765477786913079296 for /10.117.199.234 INFO 21:11:11,699 Deleting any stored hints for 10.117.199.234 INFO 21:11:11,701 Announcing that I have left the ring for 3ms INFO 21:11:11,702 Enqueuing flush of Memtable-HintsColumnFamily@1635064453(0/0 serialized/live bytes, 1 ops) INFO 21:11:11,703 Writing Memtable-HintsColumnFamily@1635064453(0/0 serialized/live bytes, 1 ops) INFO 21:11:11,729 Completed flushing /var/lib/cassandra/data/system/HintsColumnFamily-g-1-Data.db (64 bytes) INFO 21:11:41,702 Shutting down MessageService... INFO 21:11:41,704 Shutdown complete (no further commands will be processed) INFO 21:11:41,704 MessagingService shutting down server thread. INFO 21:11:41,706 Decommissioned ERROR 21:11:44,261 Fatal exception in thread Thread[Thread-9632,5,main] java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut down at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:60) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658) at org.apache.cassandra.net.MessagingService.receive(MessagingService.java:490) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:133) INFO 21:43:22,198 Cassandra shutting down... INFO 21:43:22,199 Stop listening to thrift clients 2 last lines are about stopping cassandra via init.d script. Is that exception normal? If it is, then it should be not exception but normal operation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3232) Startup after abrupt shutdown may result in failure to delete a temporary SSTable
[
https://issues.apache.org/jira/browse/CASSANDRA-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108596#comment-13108596
]
Jonathan Ellis commented on CASSANDRA-3232:
---
Sounds like Windows hasn't realized the old owner is dead yet and thus refuses
the delete?
Startup after abrupt shutdown may result in failure to delete a temporary
SSTable
-
Key: CASSANDRA-3232
URL: https://issues.apache.org/jira/browse/CASSANDRA-3232
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.5
Environment: Windows 7 64bit
Reporter: Ophir Radnitz
Priority: Minor
After killing the server and starting it up again, once in 3-4 times we're
encountering the described problem.
A relevant stack trace:
{noformat}
11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see
http://wiki.apache.org/cassandra/MemtableSSTable)
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[cassandra_server,5,main]
java.io.IOError: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
at
org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
at
org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
at
com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
... 6 more
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[FlushWriter:1,5,main]
java.io.IOError: java.io.IOException: rename failed of
c:\cassandra\data\system\LocationInfo-g-24-Index.db
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: rename failed of
c:\cassandra\data\system\LocationInfo-g-24-Index.db
at
org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
... 9 more
11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
11:10:44 SSTableReader [INFO] Opening
c:\cassandra\data\20110914\session_hits-g-456
...
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1173091 - /cassandra/branches/cassandra-1.0.0/build.xml
Author: eevans
Date: Tue Sep 20 11:38:42 2011
New Revision: 1173091
URL: http://svn.apache.org/viewvc?rev=1173091view=rev
Log:
copy clientutil jar to binary release artifacts
Patch by eevans; reviewed by jbellis for CASSANDRA-3230
Modified:
cassandra/branches/cassandra-1.0.0/build.xml
Modified: cassandra/branches/cassandra-1.0.0/build.xml
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/build.xml?rev=1173091r1=1173090r2=1173091view=diff
==
--- cassandra/branches/cassandra-1.0.0/build.xml (original)
+++ cassandra/branches/cassandra-1.0.0/build.xml Tue Sep 20 11:38:42 2011
@@ -839,6 +839,7 @@ url=${svn.entry.url}?pathrev=${svn.entry
fileset dir=${build.dir}
include name=${final.name}.jar /
include name=${ant.project.name}-thrift-${version}.jar /
+ include name=${ant.project.name}-clientutil-${version}.jar /
/fileset
/copy
copy todir=${dist.dir}/javadoc
[jira] [Commented] (CASSANDRA-3232) Startup after abrupt shutdown may result in failure to delete a temporary SSTable
[
https://issues.apache.org/jira/browse/CASSANDRA-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108608#comment-13108608
]
Ophir Radnitz commented on CASSANDRA-3232:
--
Very possible, although upon further checking in turns out the file that failed
to be deleted does not exist. Could this mean the storage is somehow out of
sync with the file system?
Moreover, Cassandra fails to load after such failure. I think there's a point
to be made about wrapping the call to ColumnFamilyStore#scrubDataDirectories in
AbstractCassandraDaemon#setup, logging errors and continuing to load anyway.
Startup after abrupt shutdown may result in failure to delete a temporary
SSTable
-
Key: CASSANDRA-3232
URL: https://issues.apache.org/jira/browse/CASSANDRA-3232
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.5
Environment: Windows 7 64bit
Reporter: Ophir Radnitz
Priority: Minor
After killing the server and starting it up again, once in 3-4 times we're
encountering the described problem.
A relevant stack trace:
{noformat}
11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see
http://wiki.apache.org/cassandra/MemtableSSTable)
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[cassandra_server,5,main]
java.io.IOError: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
at
org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
at
org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
at
com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
... 6 more
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[FlushWriter:1,5,main]
java.io.IOError: java.io.IOException: rename failed of
c:\cassandra\data\system\LocationInfo-g-24-Index.db
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: rename failed of
c:\cassandra\data\system\LocationInfo-g-24-Index.db
at
org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
... 9 more
11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
11:10:44 SSTableReader [INFO] Opening
c:\cassandra\data\20110914\session_hits-g-456
...
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3232) Startup after abrupt shutdown may result in failure to delete a temporary SSTable
[
https://issues.apache.org/jira/browse/CASSANDRA-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108608#comment-13108608
]
Ophir Radnitz edited comment on CASSANDRA-3232 at 9/20/11 11:43 AM:
Very possible, although upon further checking it turns out the file that failed
to be deleted does not exist. Could this mean the storage is somehow out of
sync with the file system?
Moreover, Cassandra fails to load after such failure. I think there's a point
to be made about wrapping the call to ColumnFamilyStore#scrubDataDirectories in
AbstractCassandraDaemon#setup, logging errors and continuing to load anyway.
was (Author: liqweed):
Very possible, although upon further checking in turns out the file that
failed to be deleted does not exist. Could this mean the storage is somehow out
of sync with the file system?
Moreover, Cassandra fails to load after such failure. I think there's a point
to be made about wrapping the call to ColumnFamilyStore#scrubDataDirectories in
AbstractCassandraDaemon#setup, logging errors and continuing to load anyway.
Startup after abrupt shutdown may result in failure to delete a temporary
SSTable
-
Key: CASSANDRA-3232
URL: https://issues.apache.org/jira/browse/CASSANDRA-3232
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.5
Environment: Windows 7 64bit
Reporter: Ophir Radnitz
Priority: Minor
After killing the server and starting it up again, once in 3-4 times we're
encountering the described problem.
A relevant stack trace:
{noformat}
11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see
http://wiki.apache.org/cassandra/MemtableSSTable)
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[cassandra_server,5,main]
java.io.IOError: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
at
org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
at
org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
at
com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
... 6 more
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[FlushWriter:1,5,main]
java.io.IOError: java.io.IOException: rename failed of
c:\cassandra\data\system\LocationInfo-g-24-Index.db
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: rename failed of
c:\cassandra\data\system\LocationInfo-g-24-Index.db
at
org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
... 9 more
11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
11:10:44 SSTableReader [INFO] Opening
c:\cassandra\data\20110914\session_hits-g-456
...
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3232) Startup after abrupt shutdown may result in failure to delete a temporary SSTable
[
https://issues.apache.org/jira/browse/CASSANDRA-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108615#comment-13108615
]
Ophir Radnitz commented on CASSANDRA-3232:
--
I neglected to mention a probably important detail - I'm running an embedded
Cassandra instance using EmbeddedCassandraService and do my writes via the
StorageProxy API to avoid thrift serialization.
Startup after abrupt shutdown may result in failure to delete a temporary
SSTable
-
Key: CASSANDRA-3232
URL: https://issues.apache.org/jira/browse/CASSANDRA-3232
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.5
Environment: Windows 7 64bit
Reporter: Ophir Radnitz
Priority: Minor
After killing the server and starting it up again, once in 3-4 times we're
encountering the described problem.
A relevant stack trace:
{noformat}
11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see
http://wiki.apache.org/cassandra/MemtableSSTable)
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[cassandra_server,5,main]
java.io.IOError: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
at
org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
at
org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
at
com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Failed to delete
c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
... 6 more
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread
Thread[FlushWriter:1,5,main]
java.io.IOError: java.io.IOException: rename failed of
c:\cassandra\data\system\LocationInfo-g-24-Index.db
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: rename failed of
c:\cassandra\data\system\LocationInfo-g-24-Index.db
at
org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
... 9 more
11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
11:10:44 SSTableReader [INFO] Opening
c:\cassandra\data\20110914\session_hits-g-456
...
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2434) range movements can violate consistency
[
https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108617#comment-13108617
]
Jonathan Ellis commented on CASSANDRA-2434:
---
bq. It's always been unsupported to bootstrap a second node into the same
token arc while a previous one is ongoing.
I'm pretty sure now that this is incorrect; we fixed it back in CASSANDRA-603.
I'm updating the comments in TokenMetadata as follows:
{noformat}
// Prior to CASSANDRA-603, we just had ttMapRange, InetAddress
pendingRangestt,
// which was added to when a node began bootstrap and removed from when it
finished.
//
// This is inadequate when multiple changes are allowed simultaneously.
For example,
// suppose that there is a ring of nodes A, C and E, with replication
factor 3.
// Node D bootstraps between C and E, so its pending ranges will be E-A,
A-C and C-D.
// Now suppose node B bootstraps between A and C at the same time. Its
pending ranges
// would be C-E, E-A and A-B. Now both nodes need to be assigned pending
range E-A,
// which we would be unable to represent with the old Map. The same thing
happens
// even more obviously for any nodes that boot simultaneously between same
two nodes.
//
// So, we made two changes:
//
// First, we changed pendingRanges to a ttMultimapRange,
InetAddress/tt (now
// ttMapString, MultimapRange, InetAddress/tt, because replication
strategy
// and options are per-KeySpace).
//
// Second, we added the bootstrapTokens and leavingEndpoints collections,
so we can
// rebuild pendingRanges from the complete information of what is going on,
when
// additional changes are made mid-operation.
//
// Finally, note that recording the tokens of joining nodes in
bootstrapTokens also
// means we can detect and reject the addition of multiple nodes at the
same token
// before one becomes part of the ring.
private BiMapToken, InetAddress bootstrapTokens = HashBiMap.create();
// (don't need to record Token here since it's still part of
tokenToEndpointMap until it's done leaving)
private SetInetAddress leavingEndpoints = new HashSetInetAddress();
// this is a cache of the calculation from {tokenToEndpointMap,
bootstrapTokens, leavingEndpoints}
private ConcurrentMapString, MultimapRange, InetAddress pendingRanges =
new ConcurrentHashMapString, MultimapRange, InetAddress();
{noformat}
range movements can violate consistency
---
Key: CASSANDRA-2434
URL: https://issues.apache.org/jira/browse/CASSANDRA-2434
Project: Cassandra
Issue Type: Bug
Reporter: Peter Schuller
Assignee: paul cannon
Fix For: 1.0.1
Attachments: 2434-3.patch.txt, 2434-testery.patch.txt
My reading (a while ago) of the code indicates that there is no logic
involved during bootstrapping that avoids consistency level violations. If I
recall correctly it just grabs neighbors that are currently up.
There are at least two issues I have with this behavior:
* If I have a cluster where I have applications relying on QUORUM with RF=3,
and bootstrapping complete based on only one node, I have just violated the
supposedly guaranteed consistency semantics of the cluster.
* Nodes can flap up and down at any time, so even if a human takes care to
look at which nodes are up and things about it carefully before
bootstrapping, there's no guarantee.
A complication is that not only does it depend on use-case where this is an
issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster
which is otherwise used for QUORUM operations you may wish to accept
less-than-quorum nodes during bootstrap in various emergency situations.
A potential easy fix is to have bootstrap take an argument which is the
number of hosts to bootstrap from, or to assume QUORUM if none is given.
(A related concern is bootstrapping across data centers. You may *want* to
bootstrap to a local node and then do a repair to avoid sending loads of data
across DC:s while still achieving consistency. Or even if you don't care
about the consistency issues, I don't think there is currently a way to
bootstrap from local nodes only.)
Thoughts?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1173099 - /cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/locator/TokenMetadata.java
Author: jbellis
Date: Tue Sep 20 11:59:45 2011
New Revision: 1173099
URL: http://svn.apache.org/viewvc?rev=1173099view=rev
Log:
update comments
Modified:
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/locator/TokenMetadata.java
Modified:
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/locator/TokenMetadata.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/locator/TokenMetadata.java?rev=1173099r1=1173098r2=1173099view=diff
==
---
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/locator/TokenMetadata.java
(original)
+++
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/locator/TokenMetadata.java
Tue Sep 20 11:59:45 2011
@@ -43,22 +43,34 @@ public class TokenMetadata
/* Maintains token to endpoint map of every node in the cluster. */
private BiMapToken, InetAddress tokenToEndpointMap;
-// Suppose that there is a ring of nodes A, C and E, with replication
factor 3.
+// Prior to CASSANDRA-603, we just had ttMapRange, InetAddress
pendingRangestt,
+// which was added to when a node began bootstrap and removed from when it
finished.
+//
+// This is inadequate when multiple changes are allowed simultaneously.
For example,
+// suppose that there is a ring of nodes A, C and E, with replication
factor 3.
// Node D bootstraps between C and E, so its pending ranges will be E-A,
A-C and C-D.
-// Now suppose node B bootstraps between A and C at the same time. Its
pending ranges would be C-E, E-A and A-B.
-// Now both nodes have pending range E-A in their list, which will cause
pending range collision
-// even though we're only talking about replica range, not even primary
range. The same thing happens
-// for any nodes that boot simultaneously between same two nodes. For this
we cannot simply make pending ranges a ttMultimap/tt,
-// since that would make us unable to notice the real problem of two nodes
trying to boot using the same token.
-// In order to do this properly, we need to know what tokens are booting
at any time.
+// Now suppose node B bootstraps between A and C at the same time. Its
pending ranges
+// would be C-E, E-A and A-B. Now both nodes need to be assigned pending
range E-A,
+// which we would be unable to represent with the old Map. The same thing
happens
+// even more obviously for any nodes that boot simultaneously between same
two nodes.
+//
+// So, we made two changes:
+//
+// First, we changed pendingRanges to a ttMultimapRange,
InetAddress/tt (now
+// ttMapString, MultimapRange, InetAddress/tt, because replication
strategy
+// and options are per-KeySpace).
+//
+// Second, we added the bootstrapTokens and leavingEndpoints collections,
so we can
+// rebuild pendingRanges from the complete information of what is going
on, when
+// additional changes are made mid-operation.
+//
+// Finally, note that recording the tokens of joining nodes in
bootstrapTokens also
+// means we can detect and reject the addition of multiple nodes at the
same token
+// before one becomes part of the ring.
private BiMapToken, InetAddress bootstrapTokens = HashBiMap.create();
-
-// we will need to know at all times what nodes are leaving and calculate
ranges accordingly.
-// An anonymous pending ranges list is not enough, as that does not tell
which node is leaving
-// and/or if the ranges are there because of bootstrap or leave operation.
-// (See CASSANDRA-603 for more detail + examples).
+// (don't need to record Token here since it's still part of
tokenToEndpointMap until it's done leaving)
private SetInetAddress leavingEndpoints = new HashSetInetAddress();
-
+// this is a cache of the calculation from {tokenToEndpointMap,
bootstrapTokens, leavingEndpoints}
private ConcurrentMapString, MultimapRange, InetAddress pendingRanges
= new ConcurrentHashMapString, MultimapRange, InetAddress();
// nodes which are migrating to the new tokens in the ring
[jira] [Commented] (CASSANDRA-3233) fatar exception after decommission
[ https://issues.apache.org/jira/browse/CASSANDRA-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108628#comment-13108628 ] Zenek Kraweznik commented on CASSANDRA-3233: ERROR 21:11:44,261 Fatal exception in thread - ERROR 21:11:44,261 Harmless exception in thread :] don't threat users :P fatar exception after decommission -- Key: CASSANDRA-3233 URL: https://issues.apache.org/jira/browse/CASSANDRA-3233 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.5 Environment: linux 2.6.32 (openvz, debian) java sun 1.6.0_26: sun-java6-bin 6.26-0squeeze1 sun-java6-jre 6.26-0squeeze1 Reporter: Zenek Kraweznik after decommission node (succesful) i found in logfile information about fatal exception: INFO 20:19:16,753 Streaming to /10.117.199.233 INFO 21:11:11,699 Removing token 127605887595351923798765477786913079296 for /10.117.199.234 INFO 21:11:11,699 Deleting any stored hints for 10.117.199.234 INFO 21:11:11,701 Announcing that I have left the ring for 3ms INFO 21:11:11,702 Enqueuing flush of Memtable-HintsColumnFamily@1635064453(0/0 serialized/live bytes, 1 ops) INFO 21:11:11,703 Writing Memtable-HintsColumnFamily@1635064453(0/0 serialized/live bytes, 1 ops) INFO 21:11:11,729 Completed flushing /var/lib/cassandra/data/system/HintsColumnFamily-g-1-Data.db (64 bytes) INFO 21:11:41,702 Shutting down MessageService... INFO 21:11:41,704 Shutdown complete (no further commands will be processed) INFO 21:11:41,704 MessagingService shutting down server thread. INFO 21:11:41,706 Decommissioned ERROR 21:11:44,261 Fatal exception in thread Thread[Thread-9632,5,main] java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut down at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:60) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658) at org.apache.cassandra.net.MessagingService.receive(MessagingService.java:490) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:133) INFO 21:43:22,198 Cassandra shutting down... INFO 21:43:22,199 Stop listening to thrift clients 2 last lines are about stopping cassandra via init.d script. Is that exception normal? If it is, then it should be not exception but normal operation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3224) LeveledCompactionStrategy is too complacent
[
https://issues.apache.org/jira/browse/CASSANDRA-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108637#comment-13108637
]
Brandon Williams commented on CASSANDRA-3224:
-
+1
LeveledCompactionStrategy is too complacent
---
Key: CASSANDRA-3224
URL: https://issues.apache.org/jira/browse/CASSANDRA-3224
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.0
Reporter: Brandon Williams
Assignee: Jonathan Ellis
Labels: compaction
Fix For: 1.0.0
Attachments: 3224-v2.txt, 3224-v3.txt, 3224-v4.txt, 3224-v5.txt,
3224-v6.txt, 3224.txt, system.log.bz2
As the title says, it barely does anything. I inserted 50G worth of data
with 1G heap and 99% overwrite ratio, and it only compacted twice:
{noformat}
INFO [CompactionExecutor:1] 2011-09-16 22:29:54,572 CompactionTask.java
(line 118) Compacting
[SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-1-Data.db')]
INFO [CompactionExecutor:1] 2011-09-16 22:29:58,606 CompactionTask.java
(line 220) Compacted to
[/var/lib/cassandra/data/Keyspace1/Standard1-h-2-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-4-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-5-Data.db,].
12,595,811 to 12,595,811 (~100% of original) bytes for 40,501 keys at
3.058122MBPS. Time: 3,928ms.
INFO [CompactionExecutor:1] 2011-09-16 22:29:58,607 CompactionTask.java
(line 222) CF Total Bytes Compacted: 12,595,811
INFO [CompactionExecutor:3] 2011-09-16 22:29:58,889 CompactionTask.java
(line 118) Compacting
[SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-4-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-2-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-5-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-3-Data.db')]
INFO [CompactionExecutor:3] 2011-09-16 22:30:06,900 CompactionTask.java
(line 220) Compacted to
[/var/lib/cassandra/data/Keyspace1/Standard1-h-7-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-9-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-11-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-12-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-14-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-15-Data.db,].
28,374,396 to 28,374,396 (~100% of original) bytes for 91,236 keys at
3.380379MBPS. Time: 8,005ms.
INFO [CompactionExecutor:3] 2011-09-16 22:30:06,901 CompactionTask.java
(line 222) CF Total Bytes Compacted: 40,970,207
{noformat}
Resulting in the following levels:
{noformat}
L0: 4965
L1: 6
L2: 0
L3: 0
L4: 0
L5: 0
L6: 0
L7: 0
{noformat}
This is obviously going to result in extremely poor read performance.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1173131 - in /cassandra/branches/cassandra-1.0.0: ./ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/compaction/
Author: jbellis
Date: Tue Sep 20 13:13:04 2011
New Revision: 1173131
URL: http://svn.apache.org/viewvc?rev=1173131view=rev
Log:
LeveledCompactionStrategy fixes
patch by jbellis; tested by brandonwilliams for CASSANDRA-3224
Modified:
cassandra/branches/cassandra-1.0.0/CHANGES.txt
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
Modified: cassandra/branches/cassandra-1.0.0/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/CHANGES.txt?rev=1173131r1=1173130r2=1173131view=diff
==
--- cassandra/branches/cassandra-1.0.0/CHANGES.txt (original)
+++ cassandra/branches/cassandra-1.0.0/CHANGES.txt Tue Sep 20 13:13:04 2011
@@ -10,6 +10,8 @@
true) and set default badness threshold to 0.1 (CASSANDRA-3229)
* Base choice of random or balanced token on bootstrap on whether
schema definitions were found (CASSANDRA-3219)
+ * Fixes for LeveledCompactionStrategy score computation, prioritization,
+ and scheduling (CASSANDRA-3224)
1.0.0-beta1
Modified:
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1173131r1=1173130r2=1173131view=diff
==
---
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
(original)
+++
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Tue Sep 20 13:13:04 2011
@@ -328,7 +328,7 @@ public class ColumnFamilyStore implement
*/
public static void scrubDataDirectories(String table, String columnFamily)
{
-logger.info(Removing compacted SSTable files (see
http://wiki.apache.org/cassandra/MemtableSSTable));
+logger.info(Removing compacted SSTable files from + columnFamily +
(see http://wiki.apache.org/cassandra/MemtableSSTable));
for (Map.EntryDescriptor,SetComponent sstableFiles : files(table,
columnFamily, true, true).entrySet())
{
Descriptor desc = sstableFiles.getKey();
Modified:
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java?rev=1173131r1=1173130r2=1173131view=diff
==
---
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
(original)
+++
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
Tue Sep 20 13:13:04 2011
@@ -76,9 +76,20 @@ public class LeveledCompactionStrategy e
logger.info(this + subscribed to the data tracker.);
manifest = LeveledManifest.create(cfs, this.maxSSTableSize);
+logger.debug(Created {}, manifest);
// override min/max for this strategy
cfs.setMaximumCompactionThreshold(Integer.MAX_VALUE);
cfs.setMinimumCompactionThreshold(1);
+
+// TODO this is redundant wrt the kickoff in
AbstractCompactionStrategy, once CASSANDRA-X is done
+Runnable runnable = new Runnable()
+{
+public void run()
+{
+
CompactionManager.instance.submitBackground(LeveledCompactionStrategy.this.cfs);
+}
+};
+StorageService.optionalTasks.scheduleAtFixedRate(runnable, 5 * 60, 5,
TimeUnit.SECONDS);
}
public void shutdown()
@@ -96,12 +107,17 @@ public class LeveledCompactionStrategy e
{
LeveledCompactionTask currentTask = task.get();
if (currentTask != null !currentTask.isDone())
+{
+logger.debug(Compaction still in progress for {}, this);
return Collections.emptyList();
+}
CollectionSSTableReader sstables =
manifest.getCompactionCandidates();
-logger.debug(CompactionManager candidates are {},
StringUtils.join(sstables, ,));
if (sstables.isEmpty())
+{
+logger.debug(No compaction necessary for {}, this);
return Collections.emptyList();
+}
LeveledCompactionTask newTask = new LeveledCompactionTask(cfs,
sstables, gcBefore, this.maxSSTableSize);
return task.compareAndSet(currentTask, newTask)
@@ -139,4 +155,10 @@ public class LeveledCompactionStrategy e
manifest.logDistribution();
}
}
+
+@Override
+
svn commit: r1173134 - in /cassandra/branches/cassandra-1.0: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/compa
Author: jbellis
Date: Tue Sep 20 13:19:22 2011
New Revision: 1173134
URL: http://svn.apache.org/viewvc?rev=1173134view=rev
Log:
merge #3224 from 1.0.0
Modified:
cassandra/branches/cassandra-1.0/ (props changed)
cassandra/branches/cassandra-1.0/CHANGES.txt
cassandra/branches/cassandra-1.0/build.xml
cassandra/branches/cassandra-1.0/contrib/ (props changed)
cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
(props changed)
cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
(props changed)
cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
(props changed)
cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
(props changed)
cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
(props changed)
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/locator/TokenMetadata.java
Propchange: cassandra/branches/cassandra-1.0/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Sep 20 13:19:22 2011
@@ -5,7 +5,7 @@
/cassandra/branches/cassandra-0.8.0:1125021-1130369
/cassandra/branches/cassandra-0.8.1:1101014-1125018
/cassandra/branches/cassandra-1.0:1167106,1167185
-/cassandra/branches/cassandra-1.0.0:1167104-1172718
+/cassandra/branches/cassandra-1.0.0:1167104-1173133
/cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689
/cassandra/tags/cassandra-0.8.0-rc1:1102511-1125020
/cassandra/trunk:1167085-1167102,1169870
Modified: cassandra/branches/cassandra-1.0/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/CHANGES.txt?rev=1173134r1=1173133r2=1173134view=diff
==
--- cassandra/branches/cassandra-1.0/CHANGES.txt (original)
+++ cassandra/branches/cassandra-1.0/CHANGES.txt Tue Sep 20 13:19:22 2011
@@ -14,6 +14,8 @@
true) and set default badness threshold to 0.1 (CASSANDRA-3229)
* Base choice of random or balanced token on bootstrap on whether
schema definitions were found (CASSANDRA-3219)
+ * Fixes for LeveledCompactionStrategy score computation, prioritization,
+ and scheduling (CASSANDRA-3224)
1.0.0-beta1
Modified: cassandra/branches/cassandra-1.0/build.xml
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/build.xml?rev=1173134r1=1173133r2=1173134view=diff
==
--- cassandra/branches/cassandra-1.0/build.xml (original)
+++ cassandra/branches/cassandra-1.0/build.xml Tue Sep 20 13:19:22 2011
@@ -839,6 +839,7 @@ url=${svn.entry.url}?pathrev=${svn.entry
fileset dir=${build.dir}
include name=${final.name}.jar /
include name=${ant.project.name}-thrift-${version}.jar /
+ include name=${ant.project.name}-clientutil-${version}.jar /
/fileset
/copy
copy todir=${dist.dir}/javadoc
Propchange: cassandra/branches/cassandra-1.0/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Sep 20 13:19:22 2011
@@ -5,7 +5,7 @@
/cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369
/cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018
/cassandra/branches/cassandra-1.0/contrib:1167106,1167185
-/cassandra/branches/cassandra-1.0.0/contrib:1167104-1172718
+/cassandra/branches/cassandra-1.0.0/contrib:1167104-1173133
/cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689
/cassandra/tags/cassandra-0.8.0-rc1/contrib:1102511-1125020
/cassandra/trunk/contrib:1167085-1167102,1169870
Propchange:
cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Sep 20 13:19:22 2011
@@ -5,7 +5,7 @@
/cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369
/cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018
/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167106,1167185
-/cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167104-1172718
svn commit: r1173136 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/compaction/ src/java/o
Author: jbellis
Date: Tue Sep 20 13:20:28 2011
New Revision: 1173136
URL: http://svn.apache.org/viewvc?rev=1173136view=rev
Log:
merge #3224 from 1.0
Modified:
cassandra/trunk/ (props changed)
cassandra/trunk/CHANGES.txt
cassandra/trunk/build.xml
cassandra/trunk/contrib/ (props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
(props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
(props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
(props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
(props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
(props changed)
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
cassandra/trunk/src/java/org/apache/cassandra/locator/TokenMetadata.java
Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Sep 20 13:20:28 2011
@@ -4,8 +4,8 @@
/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1171737,1172026,1172591
/cassandra/branches/cassandra-0.8.0:1125021-1130369
/cassandra/branches/cassandra-0.8.1:1101014-1125018
-/cassandra/branches/cassandra-1.0:1167085-1172719
-/cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1172718
+/cassandra/branches/cassandra-1.0:1167085-1173134
+/cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1173133
/cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689
/cassandra/tags/cassandra-0.8.0-rc1:1102511-1125020
/incubator/cassandra/branches/cassandra-0.3:774578-796573
Modified: cassandra/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1173136r1=1173135r2=1173136view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Tue Sep 20 13:20:28 2011
@@ -14,6 +14,8 @@
true) and set default badness threshold to 0.1 (CASSANDRA-3229)
* Base choice of random or balanced token on bootstrap on whether
schema definitions were found (CASSANDRA-3219)
+ * Fixes for LeveledCompactionStrategy score computation, prioritization,
+ and scheduling (CASSANDRA-3224)
1.0.0-beta1
Modified: cassandra/trunk/build.xml
URL:
http://svn.apache.org/viewvc/cassandra/trunk/build.xml?rev=1173136r1=1173135r2=1173136view=diff
==
--- cassandra/trunk/build.xml (original)
+++ cassandra/trunk/build.xml Tue Sep 20 13:20:28 2011
@@ -881,6 +881,7 @@ url=${svn.entry.url}?pathrev=${svn.entry
fileset dir=${build.dir}
include name=${final.name}.jar /
include name=${ant.project.name}-thrift-${version}.jar /
+ include name=${ant.project.name}-clientutil-${version}.jar /
/fileset
/copy
copy todir=${dist.dir}/javadoc
Propchange: cassandra/trunk/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Sep 20 13:20:28 2011
@@ -4,8 +4,8 @@
/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1171737,1172026,1172591
/cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369
/cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018
-/cassandra/branches/cassandra-1.0/contrib:1167085-1172719
-/cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1172718
+/cassandra/branches/cassandra-1.0/contrib:1167085-1173134
+/cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1173133
/cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689
/cassandra/tags/cassandra-0.8.0-rc1/contrib:1102511-1125020
/incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573
Propchange:
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Sep 20 13:20:28 2011
@@ -4,8 +4,8 @@
/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1171737,1172026,1172591
/cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369
/cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018
-/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167085-1172719
[jira] [Commented] (CASSANDRA-3140) Expose server, api versions to CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108744#comment-13108744 ] satish babu krishnamoorthy commented on CASSANDRA-3140: --- Can i change the implementation to support cqlshsystem server_version cqlshsystem api_version Satish Expose server, api versions to CQL -- Key: CASSANDRA-3140 URL: https://issues.apache.org/jira/browse/CASSANDRA-3140 Project: Cassandra Issue Type: New Feature Reporter: Jonathan Ellis Assignee: satish babu krishnamoorthy Priority: Minor Fix For: 1.0.1 Attachments: 3140-patch.diff Need to expose the CQL api version; might as well include the server version while we're at it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2434) range movements can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108793#comment-13108793 ] paul cannon commented on CASSANDRA-2434: bq. I'm pretty sure now that this is incorrect; Well, doh. That puts us back at my first question: bq. So, it looks like it will be possible for the node-that-will-be-removed to change between starting a bootstrap and finishing it (other nodes being bootstrapped/moved/decom'd during that time period); in some cases, that could still lead to a consistency violation. Is that unlikely enough that we don't care, here? At least the situation would be better with the proposed fix than it is now. range movements can violate consistency --- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: paul cannon Fix For: 1.0.1 Attachments: 2434-3.patch.txt, 2434-testery.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2434) range movements can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108800#comment-13108800 ] Jonathan Ellis commented on CASSANDRA-2434: --- Can you give an example for illustration? range movements can violate consistency --- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: paul cannon Fix For: 1.0.1 Attachments: 2434-3.patch.txt, 2434-testery.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of ArticlesAndPresentations by DavidJeferson
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The ArticlesAndPresentations page has been changed by DavidJeferson: http://wiki.apache.org/cassandra/ArticlesAndPresentations?action=diffrev1=125rev2=126 * [[http://blog.octo.com/en/nosql-lets-play-with-cassandra-part-13|Cassandra Overview]], [[http://blog.octo.com/en/nosql-lets-play-with-cassandra-part-23|Setup]] and [[http://blog.octo.com/en/nosql-lets-play-with-cassandra-part-13|Java Thrift Client]] : [[http://blog.octo.com/en/nosql-lets-play-with-cassandra-part-13/|Part1]], [[http://blog.octo.com/en/nosql-lets-play-with-cassandra-part-23/|Part2]] and [[http://blog.octo.com/en/nosql-lets-play-with-cassandra-part-33/|Part3]], June 2010 * [[http://ria101.wordpress.com/2010/06/11/pelops-the-beautiful-cassandra-database-client-for-java/|Up and running quickly in Java using Pelops]], June 2010 * [[http://anismiles.wordpress.com/2010/05/27/lucandra-an-inside-story/|Lucandra: an inside story]] + * [[http://myinternetmarketingservices.com/|Internet Marketing Services]] * [[http://io.typepad.com/glossary.html|A Cassandra Glossary]] (simple explanation of key terms) * [[http://ria101.wordpress.com/2010/05/12/locking-and-transactions-over-cassandra-using-cages/|Locking and transactions over Cassandra using Cages]], May 2010 * [[http://cgpad.com/Cassandra%20in%20Action%20with%20Twitter%E2%80%99s%20Ruby%20Client.pdf|Cassandra in Action with the Ruby Client]]
[Cassandra Wiki] Trivial Update of ArticlesAndPresentations by gdusbabek
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The ArticlesAndPresentations page has been changed by gdusbabek: http://wiki.apache.org/cassandra/ArticlesAndPresentations?action=diffrev1=126rev2=127 Comment: revert spam * [[http://blog.octo.com/en/nosql-lets-play-with-cassandra-part-13|Cassandra Overview]], [[http://blog.octo.com/en/nosql-lets-play-with-cassandra-part-23|Setup]] and [[http://blog.octo.com/en/nosql-lets-play-with-cassandra-part-13|Java Thrift Client]] : [[http://blog.octo.com/en/nosql-lets-play-with-cassandra-part-13/|Part1]], [[http://blog.octo.com/en/nosql-lets-play-with-cassandra-part-23/|Part2]] and [[http://blog.octo.com/en/nosql-lets-play-with-cassandra-part-33/|Part3]], June 2010 * [[http://ria101.wordpress.com/2010/06/11/pelops-the-beautiful-cassandra-database-client-for-java/|Up and running quickly in Java using Pelops]], June 2010 * [[http://anismiles.wordpress.com/2010/05/27/lucandra-an-inside-story/|Lucandra: an inside story]] - * [[http://myinternetmarketingservices.com/|Internet Marketing Services]] * [[http://io.typepad.com/glossary.html|A Cassandra Glossary]] (simple explanation of key terms) * [[http://ria101.wordpress.com/2010/05/12/locking-and-transactions-over-cassandra-using-cages/|Locking and transactions over Cassandra using Cages]], May 2010 * [[http://cgpad.com/Cassandra%20in%20Action%20with%20Twitter%E2%80%99s%20Ruby%20Client.pdf|Cassandra in Action with the Ruby Client]]
[jira] [Commented] (CASSANDRA-2863) NPE when writing SSTable generated via repair
[ https://issues.apache.org/jira/browse/CASSANDRA-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13109079#comment-13109079 ] chris erway commented on CASSANDRA-2863: I just got a similar-looking NPE stack trace. The node that raised the exception was receiving streams from a node being decommissioned (with nodetool decommission). Both nodes were running 0.8.6; both had been upgraded a few hours earlier from 0.8.4. The first: ERROR [CompactionExecutor:72] 2011-09-20 22:34:20,892 AbstractCassandraDaemon.java (line 139) Fatal exception in thread Thread[CompactionExecutor:72,1,main] java.lang.NullPointerException at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382) at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370) at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315) at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1108) at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1099) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Then half a minute later: INFO [CompactionExecutor:72] 2011-09-20 22:34:46,923 SSTableReader.java (line 162) Opening /mnt/cassandra/data/Tracelytics/Events-g-1536 ERROR [Thread-785] 2011-09-20 22:34:52,054 AbstractCassandraDaemon.java (line 139) Fatal exception in thread Thread[Thread-785,5,main] java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NullPointerException at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:154) at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63) at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:189) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:117) Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) at java.util.concurrent.FutureTask.get(FutureTask.java:83) at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:138) ... 3 more Caused by: java.lang.NullPointerException at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382) at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370) at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315) at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1108) at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1099) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) NPE when writing SSTable generated via repair - Key: CASSANDRA-2863 URL: https://issues.apache.org/jira/browse/CASSANDRA-2863 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.1 Reporter: Héctor Izquierdo A NPE is generated during repair when closing an sstable generated via SSTable build. It doesn't happen always. The node had been scrubbed and compacted before calling repair. INFO [CompactionExecutor:2] 2011-07-06 11:11:32,640 SSTableReader.java (line 158) Opening /d2/cassandra/data/sbs/walf-g-730 ERROR [CompactionExecutor:2] 2011-07-06 11:11:34,327 AbstractCassandraDaemon.java (line 113) Fatal exception in thread Thread[CompactionExecutor:2,1,main] java.lang.NullPointerException at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382) at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370) at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315) at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1103) at
[jira] [Issue Comment Edited] (CASSANDRA-2863) NPE when writing SSTable generated via repair
[ https://issues.apache.org/jira/browse/CASSANDRA-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13109079#comment-13109079 ] chris erway edited comment on CASSANDRA-2863 at 9/20/11 11:20 PM: -- I just got a similar-looking NPE stack trace. The node that raised the exception was receiving streams from a node being decommissioned (with nodetool decommission). Both nodes were running 0.8.6; both had been upgraded a few hours earlier from 0.8.4. The first: ERROR [CompactionExecutor:72] 2011-09-20 22:34:20,892 AbstractCassandraDaemon.java (line 139) Fatal exception in thread Thread[CompactionExecutor:72,1,main] java.lang.NullPointerException at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382) at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370) at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315) at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1108) at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1099) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Then half a minute later: INFO [CompactionExecutor:72] 2011-09-20 22:34:46,923 SSTableReader.java (line 162) Opening /mnt/cassandra/data/Keyspace/CF1-g-1536 ERROR [Thread-785] 2011-09-20 22:34:52,054 AbstractCassandraDaemon.java (line 139) Fatal exception in thread Thread[Thread-785,5,main] java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NullPointerException at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:154) at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63) at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:189) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:117) Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) at java.util.concurrent.FutureTask.get(FutureTask.java:83) at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:138) ... 3 more Caused by: java.lang.NullPointerException at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382) at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370) at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315) at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1108) at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1099) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) was (Author: cce): I just got a similar-looking NPE stack trace. The node that raised the exception was receiving streams from a node being decommissioned (with nodetool decommission). Both nodes were running 0.8.6; both had been upgraded a few hours earlier from 0.8.4. The first: ERROR [CompactionExecutor:72] 2011-09-20 22:34:20,892 AbstractCassandraDaemon.java (line 139) Fatal exception in thread Thread[CompactionExecutor:72,1,main] java.lang.NullPointerException at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382) at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370) at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315) at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1108) at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1099) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at
[jira] [Updated] (CASSANDRA-2988) Improve SSTableReader.load() when loading index files
[ https://issues.apache.org/jira/browse/CASSANDRA-2988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2988: -- Attachment: 2988-parallel-v2.txt I'll do the opposite of Pavel and start with the second. :) v2 of parallel sstable opening attached. Better code reuse, and caps the thread pool at #cores to avoid too much contention. Improve SSTableReader.load() when loading index files - Key: CASSANDRA-2988 URL: https://issues.apache.org/jira/browse/CASSANDRA-2988 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Melvin Wang Assignee: Melvin Wang Priority: Minor Fix For: 1.0.1 Attachments: 2988-parallel-v2.txt, c2988-modified-buffer.patch, c2988-parallel-load-sstables.patch * when we create BufferredRandomAccessFile, we pass skipCache=true. This hurts the read performance because we always process the index files sequentially. Simple fix would be set it to false. * multiple index files of a single column family can be loaded in parallel. This buys a lot when you have multiple super large index files. * we may also change how we buffer. By using BufferredRandomAccessFile, for every read, we need bunch of checking like - do we need to rebuffer? - isEOF()? - assertions These can be simplified to some extent. We can blindly buffer the index file by chunks and process the buffer until a key lies across boundary of a chunk. Then we rebuffer and start from the beginning of the partially read key. Conceptually, this is same as what BRAF does but w/o the overhead in the read**() methods in BRAF. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3234) LeveledCompaction has several performance problems
LeveledCompaction has several performance problems -- Key: CASSANDRA-3234 URL: https://issues.apache.org/jira/browse/CASSANDRA-3234 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 1.0.0 Two main problems: - BF size calculation doesn't take into account LCS breaking the output apart into bite sized sstables, so memory use is much higher than predicted - ManyToMany merging is slow. At least part of this is from running the full reducer machinery against single input sources, which can be optimized away. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3228) Add new range scan with clock
[ https://issues.apache.org/jira/browse/CASSANDRA-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13109100#comment-13109100 ] Ryan King commented on CASSANDRA-3228: -- the ruby gem, which is actively maintained as these things go, STILL does not have 2ary index query support Not true: https://github.com/fauna/cassandra/pull/59 Add new range scan with clock - Key: CASSANDRA-3228 URL: https://issues.apache.org/jira/browse/CASSANDRA-3228 Project: Cassandra Issue Type: New Feature Components: Core Affects Versions: 0.8.5 Reporter: Todd Nine Priority: Minor Currently, it is not possible to specify minimum clock time on columns when performing range scans. In some situations, such as custom migration or batch processing, it would be helpful to allow the client to specify a minimum clock time. This would only return columns with a clock value = the specified. I.E range scan (rowKey, startVal, endVal, revered, min clock) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3234) LeveledCompaction has several performance problems
[ https://issues.apache.org/jira/browse/CASSANDRA-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3234: -- Attachment: 0002-fix-leveled-BF-size-calculation.txt 0001-optimize-single-source-case-for-MergeIterator.txt LeveledCompaction has several performance problems -- Key: CASSANDRA-3234 URL: https://issues.apache.org/jira/browse/CASSANDRA-3234 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 0001-optimize-single-source-case-for-MergeIterator.txt, 0002-fix-leveled-BF-size-calculation.txt Two main problems: - BF size calculation doesn't take into account LCS breaking the output apart into bite sized sstables, so memory use is much higher than predicted - ManyToMany merging is slow. At least part of this is from running the full reducer machinery against single input sources, which can be optimized away. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2500) Ruby dbi client (for CQL) that conforms to AR:ConnectionAdapter
[ https://issues.apache.org/jira/browse/CASSANDRA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13109127#comment-13109127 ] Kelley Reynolds commented on CASSANDRA-2500: All of the issues that jbellis mentioned in the Sept 6th comment have been addressed with the exception of testing some of the datatypes. The datatypes tested are the ones listed here: https://github.com/apache/cassandra/blob/336476f88d107d54ef9534be1e6c0bd81e148484/doc/cql/CQL.textile#columntypes A few other people have now reviewed the code and made some minor changes along with the excellent addition of a minimal live testing framework which I did not do initially as I had thought that some other common framework was going to be set up for the various CQL drivers. I believe this to be sufficiently done. I don't know at what point this is actually going to move to the Apache Extras and who all is going to handle that but I do know that there is now a fair amount of demand for this to be released into rubygems and somebody else has already taken the trouble to release a version. Shall I go ahead and begin publishing this or shall I leave it to Cassandra Project? Ruby dbi client (for CQL) that conforms to AR:ConnectionAdapter --- Key: CASSANDRA-2500 URL: https://issues.apache.org/jira/browse/CASSANDRA-2500 Project: Cassandra Issue Type: Task Components: API Reporter: Jon Hermes Assignee: Kelley Reynolds Labels: cql Fix For: 0.8.7 Attachments: 2500.txt, genthriftrb.txt, rbcql-0.0.0.tgz Create a ruby driver for CQL. Lacking something standard (such as py-dbapi), going with something common instead -- RoR ActiveRecord Connection Adapter (http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/AbstractAdapter.html). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2434) range movements can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13109230#comment-13109230 ] Nick Bailey commented on CASSANDRA-2434: Ok, so I think there are really two consistency issues here. Firstly, picking the 'right' node to stream data to/from when making changes to the ring. Secondly, disallowing concurrent changes that have overlapping ranges. Currently we only disallow nodes from moving/decommissioning when they may potentially have data being streamed to them. There are a few examples of things we currently allow which I think are generally a bad idea. 1) Say you have nodes A and D, if you bootstrap nodes B and C at the same time in between A and D, it may turn out that the correct node to stream from for both nodes is D. Now say node C finishes bootstrapping before node B. At that point, the correct node for B to bootstrap from is technically C, although D still has the data. However, since D is no longer technically responsible for the data, the user could run cleanup on D and delete the data that B is attempting to stream. 2) The above case is also a problem when you bootstrap a node and the node it decides it needs to stream from is moving. Once that node finishes moving you could run cleanup on that node and delete data that the bootstrapping node needs. In this case, all documentation indicates you should do a cleanup after a move in order to remove old data, so it seems possibly more likely. 3) A variation of the above case is when you bootstrap a node and the node it streams from is leaving. In that case the decom may finish and the user could terminate the cassandra process and/or node breaking any streams. Not to mention the idea of a node in a decommissioned state continuing to stream seems like a bad idea. I believe it would work currently, but I'm not sure and it seems likely to break. I can't really think of any other examples but I think thats enough to illustrate that overlapping concurrent ring changes are a bad idea and we should just attempt to prevent them in all cases. An argument could be made that this would prevent you from doubling your cluster (the best way to grow) all at once, but I don't think that's really a huge deal. At most you would need RF steps to double your cluster. range movements can violate consistency --- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: paul cannon Fix For: 1.0.1 Attachments: 2434-3.patch.txt, 2434-testery.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3234) LeveledCompaction has several performance problems
[ https://issues.apache.org/jira/browse/CASSANDRA-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3234: -- Attachment: 0003-fix-leveled-BF-size-calculation.txt 0002-add-TrivialOneToOne-optimization.txt 0001-optimize-single-source-case-for-MergeIterator.txt LeveledCompaction has several performance problems -- Key: CASSANDRA-3234 URL: https://issues.apache.org/jira/browse/CASSANDRA-3234 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 0001-optimize-single-source-case-for-MergeIterator.txt, 0002-add-TrivialOneToOne-optimization.txt, 0003-fix-leveled-BF-size-calculation.txt Two main problems: - BF size calculation doesn't take into account LCS breaking the output apart into bite sized sstables, so memory use is much higher than predicted - ManyToMany merging is slow. At least part of this is from running the full reducer machinery against single input sources, which can be optimized away. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3234) LeveledCompaction has several performance problems
[ https://issues.apache.org/jira/browse/CASSANDRA-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3234: -- Attachment: (was: 0001-optimize-single-source-case-for-MergeIterator.txt) LeveledCompaction has several performance problems -- Key: CASSANDRA-3234 URL: https://issues.apache.org/jira/browse/CASSANDRA-3234 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 0001-optimize-single-source-case-for-MergeIterator.txt, 0002-add-TrivialOneToOne-optimization.txt, 0003-fix-leveled-BF-size-calculation.txt Two main problems: - BF size calculation doesn't take into account LCS breaking the output apart into bite sized sstables, so memory use is much higher than predicted - ManyToMany merging is slow. At least part of this is from running the full reducer machinery against single input sources, which can be optimized away. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3234) LeveledCompaction has several performance problems
[ https://issues.apache.org/jira/browse/CASSANDRA-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3234: -- Attachment: (was: 0002-fix-leveled-BF-size-calculation.txt) LeveledCompaction has several performance problems -- Key: CASSANDRA-3234 URL: https://issues.apache.org/jira/browse/CASSANDRA-3234 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 0001-optimize-single-source-case-for-MergeIterator.txt, 0002-add-TrivialOneToOne-optimization.txt, 0003-fix-leveled-BF-size-calculation.txt Two main problems: - BF size calculation doesn't take into account LCS breaking the output apart into bite sized sstables, so memory use is much higher than predicted - ManyToMany merging is slow. At least part of this is from running the full reducer machinery against single input sources, which can be optimized away. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3234) LeveledCompaction has several performance problems
[ https://issues.apache.org/jira/browse/CASSANDRA-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13109244#comment-13109244 ] Jonathan Ellis commented on CASSANDRA-3234: --- split OneToOne classes into separate patches and fixed bug in estimated tables calculation LeveledCompaction has several performance problems -- Key: CASSANDRA-3234 URL: https://issues.apache.org/jira/browse/CASSANDRA-3234 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 0001-optimize-single-source-case-for-MergeIterator.txt, 0002-add-TrivialOneToOne-optimization.txt, 0003-fix-leveled-BF-size-calculation.txt Two main problems: - BF size calculation doesn't take into account LCS breaking the output apart into bite sized sstables, so memory use is much higher than predicted - ManyToMany merging is slow. At least part of this is from running the full reducer machinery against single input sources, which can be optimized away. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
