[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055026#comment-13055026 ] Terje Marthinussen commented on CASSANDRA-2521: --- Did another repair overnight, one minor compaction included some 20 small sstables, all of them remains as well as a few from other compactions and the files from the repairs described before. Definitely something fishy here. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2498) Improve read performance in update-intensive workload
[ https://issues.apache.org/jira/browse/CASSANDRA-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055031#comment-13055031 ] Stu Hood commented on CASSANDRA-2498: - bq. Actually... I can't think of a good way for us to infer that (say) higher timeuuid comparators always correspond to higher column timestamps. CASSANDRA-2319 handles append-only wide row cases with an index lookup... I'm not sure they're worth worrying about for this ticket. Usecases involving slicing really need range/slice metadata to apply this type of optimization. Improve read performance in update-intensive workload - Key: CASSANDRA-2498 URL: https://issues.apache.org/jira/browse/CASSANDRA-2498 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Sylvain Lebresne Priority: Minor Labels: ponies Fix For: 1.0 Read performance in an update-heavy environment relies heavily on compaction to maintain good throughput. (This is not the case for workloads where rows are only inserted once, because the bloom filter keeps us from having to check sstables unnecessarily.) Very early versions of Cassandra attempted to mitigate this by checking sstables in descending generation order (mostly equivalent to descending mtime): once all the requested columns were found, it would not check any older sstables. This was incorrect, because data timestamp will not correspond to sstable timestamp, both because compaction has the side effect of refreshing data to a newer sstable, and because hintead handoff may send us data older than what we already have. Instead, we could create a per-sstable piece of metadata containing the most recent (client-specified) timestamp for any column in the sstable. We could then sort sstables by this timestamp instead, and perform a similar optimization (if the remaining sstable client-timestamps are older than the oldest column found in the desired result set so far, we don't need to look further). Since under almost every workload, client timestamps of data in a given sstable will tend to be similar, we expect this to cut the number of sstables down proportionally to how frequently each column in the row is updated. (If each column is updated with each write, we only have to check a single sstable.) This may also be useful information when deciding which SSTables to compact. (Note that this optimization is only appropriate for named-column queries, not slice queries, since we don't know what non-overlapping columns may exist in older sstables.) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055106#comment-13055106 ] Terje Marthinussen commented on CASSANDRA-2521: --- As for the last version of this patch, a quick look tonight shows access problems with markCurrentViewReferenced() which is called from outside cfs. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of ThirdPartySupport by DanNunan
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The ThirdPartySupport page has been changed by DanNunan:
http://wiki.apache.org/cassandra/ThirdPartySupport?action=diffrev1=12rev2=13
{{http://www.datastax.com/sites/all/themes/datastax20110201/logo.png}}
[[http://datastax.com|Datastax]] DataStax, the commercial leader in Apache
Cassandra™ offers products and services that make it easy for customers to
build, deploy and operate elastically scalable and cloud-optimized applications
and data services. The company has over 90 customers, including leaders such as
Netflix, Cisco, Rackspace and Constant Contact, and spanning verticals
including web, financial services, telecommunications, logistics and government.
- {{http://acunu.com/images/logo.png}} [[http://www.acunu.com|Acunu]] provides
software products for Cassandra applications, as well as Cassandra support and
professional services.
+ {{http://media.acunu.com/library/logo.png}} [[http://www.acunu.com|Acunu]]
provides software products for Cassandra applications, as well as Cassandra
support and professional services.
{{http://www.onzra.com/images/Small-Logo.gif}} [[http://www.ONZRA.com|ONZRA]]
has been around for over 10 years and specializes on enterprise grade
architecture, development and security consulting services utilizing many large
scale database technologies such as Cassandra, Oracle, Alegro Graph, and much
more.
[jira] [Updated] (CASSANDRA-2828) CommitLog tool
[ https://issues.apache.org/jira/browse/CASSANDRA-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Morton updated CASSANDRA-2828: Attachment: 0001-2828-07.patch LogTool command line to read 0.7 log headers and output which CF's are stopping the segment from flushing. CommitLog tool -- Key: CASSANDRA-2828 URL: https://issues.apache.org/jira/browse/CASSANDRA-2828 Project: Cassandra Issue Type: New Feature Components: Core Affects Versions: 0.7.6 Reporter: Aaron Morton Assignee: Aaron Morton Priority: Minor Attachments: 0001-2828-07.patch I wrote this for 0.7.6-2 because I had a need to see what log segment headers were preventing logs from flushing. I've not had a chance to look at it in 0.8 yet. We dont not has header files anymore, so I could turn this into a function on the StorageServiceMBean. For my use case i pulled the log headers off a server that had gone into a spin after it filled the commit log volume. nodetool was not running so these was the best solution for me. Posting here to see if there is any interest or need. I think the best approach may be to add a function to the StorageService MBean to find out which CF's are dirty in the active log segments. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2829) always flush memtables
always flush memtables
--
Key: CASSANDRA-2829
URL: https://issues.apache.org/jira/browse/CASSANDRA-2829
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.6
Reporter: Aaron Morton
Assignee: Aaron Morton
Priority: Minor
Only dirty Memtables are flushed, and so only dirty memtables are used to
discard obsolete commit log segments. This can result it log segments not been
deleted even though the data has been flushed.
Was using a 3 node 0.7.6-2 AWS cluster (DataStax AMI's) with pre 0.7 data
loaded and a running application working against the cluster. Did a rolling
restart and then kicked off a repair, one node filled up the commit log volume
with 7GB+ of log data, there was about 20 hours of log files.
{noformat}
$ sudo ls -lah commitlog/
total 6.9G
drwx-- 2 cassandra cassandra 12K 2011-06-24 20:38 .
drwxr-xr-x 3 cassandra cassandra 4.0K 2011-06-25 01:47 ..
-rw--- 1 cassandra cassandra 129M 2011-06-24 01:08
CommitLog-1308876643288.log
-rw--- 1 cassandra cassandra 28 2011-06-24 20:47
CommitLog-1308876643288.log.header
-rw-r--r-- 1 cassandra cassandra 129M 2011-06-24 01:36
CommitLog-1308877711517.log
-rw-r--r-- 1 cassandra cassandra 28 2011-06-24 20:47
CommitLog-1308877711517.log.header
-rw-r--r-- 1 cassandra cassandra 129M 2011-06-24 02:20
CommitLog-1308879395824.log
-rw-r--r-- 1 cassandra cassandra 28 2011-06-24 20:47
CommitLog-1308879395824.log.header
...
-rw-r--r-- 1 cassandra cassandra 129M 2011-06-24 20:38
CommitLog-1308946745380.log
-rw-r--r-- 1 cassandra cassandra 36 2011-06-24 20:47
CommitLog-1308946745380.log.header
-rw-r--r-- 1 cassandra cassandra 112M 2011-06-24 20:54
CommitLog-1308947888397.log
-rw-r--r-- 1 cassandra cassandra 44 2011-06-24 20:47
CommitLog-1308947888397.log.header
{noformat}
The user KS has 2 CF's with 60 minute flush times. System KS had the default
settings which is 24 hours. Will create another ticket see if these can be
reduced or if it's something users should do, in this case it would not have
mattered.
I grabbed the log headers and used the tool in CASSANDRA-2828 and most of the
segments had the system CF's marked as dirty.
{noformat}
$ bin/logtool dirty /tmp/logs/commitlog/
Not connected to a server, Keyspace and Column Family names are not available.
/tmp/logs/commitlog/CommitLog-1308876643288.log.header
Keyspace Unknown:
Cf id 0: 444
/tmp/logs/commitlog/CommitLog-1308877711517.log.header
Keyspace Unknown:
Cf id 1: 68848763
...
/tmp/logs/commitlog/CommitLog-1308944451460.log.header
Keyspace Unknown:
Cf id 1: 61074
/tmp/logs/commitlog/CommitLog-1308945597471.log.header
Keyspace Unknown:
Cf id 1000: 43175492
Cf id 1: 108483
/tmp/logs/commitlog/CommitLog-1308946745380.log.header
Keyspace Unknown:
Cf id 1000: 239223
Cf id 1: 172211
/tmp/logs/commitlog/CommitLog-1308947888397.log.header
Keyspace Unknown:
Cf id 1001: 57595560
Cf id 1: 816960
Cf id 1000: 0
{noformat}
CF 0 is the Status / LocationInfo CF and 1 is the HintedHandof CF. I dont have
it now, but IIRC CFStats showed the LocationInfo CF with dirty ops.
I was able to repo a case where flushing the CF's did not mark the log segments
as obsolete (attached unit-test patch). Steps are:
1. Write to cf1 and flush.
2. Current log segment is marked as dirty at the CL position when the flush
started, CommitLog.discardCompletedSegmentsInternal()
3. Do not write to cf1 again.
4. Roll the log, my test does this manually.
5. Write to CF2 and flush.
6. Only CF2 is flushed because it is the only dirty CF.
cfs.maybeSwitchMemtable() is not called for cf1 and so log segment 1 is still
marked as dirty from cf1.
Step 5 is not essential, just matched what I thought was happening. I thought
SystemTable.updateToken() was called which does not flush, and this was the
last thing that happened.
The expired memtable thread created by Table uses the same cfs.forceFlush()
which is a no-op if the cf or it's secondary indexes are clean.
I think the same problem would exist in 0.8.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2829) always flush memtables
[
https://issues.apache.org/jira/browse/CASSANDRA-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Morton updated CASSANDRA-2829:
Attachment: 0001-2829-unit-test.patch
0002-2829.patch
2829-unit-test contains the unit test for the problem. 2829 is the fix.
always flush memtables
--
Key: CASSANDRA-2829
URL: https://issues.apache.org/jira/browse/CASSANDRA-2829
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.6
Reporter: Aaron Morton
Assignee: Aaron Morton
Priority: Minor
Attachments: 0001-2829-unit-test.patch, 0002-2829.patch
Only dirty Memtables are flushed, and so only dirty memtables are used to
discard obsolete commit log segments. This can result it log segments not
been deleted even though the data has been flushed.
Was using a 3 node 0.7.6-2 AWS cluster (DataStax AMI's) with pre 0.7 data
loaded and a running application working against the cluster. Did a rolling
restart and then kicked off a repair, one node filled up the commit log
volume with 7GB+ of log data, there was about 20 hours of log files.
{noformat}
$ sudo ls -lah commitlog/
total 6.9G
drwx-- 2 cassandra cassandra 12K 2011-06-24 20:38 .
drwxr-xr-x 3 cassandra cassandra 4.0K 2011-06-25 01:47 ..
-rw--- 1 cassandra cassandra 129M 2011-06-24 01:08
CommitLog-1308876643288.log
-rw--- 1 cassandra cassandra 28 2011-06-24 20:47
CommitLog-1308876643288.log.header
-rw-r--r-- 1 cassandra cassandra 129M 2011-06-24 01:36
CommitLog-1308877711517.log
-rw-r--r-- 1 cassandra cassandra 28 2011-06-24 20:47
CommitLog-1308877711517.log.header
-rw-r--r-- 1 cassandra cassandra 129M 2011-06-24 02:20
CommitLog-1308879395824.log
-rw-r--r-- 1 cassandra cassandra 28 2011-06-24 20:47
CommitLog-1308879395824.log.header
...
-rw-r--r-- 1 cassandra cassandra 129M 2011-06-24 20:38
CommitLog-1308946745380.log
-rw-r--r-- 1 cassandra cassandra 36 2011-06-24 20:47
CommitLog-1308946745380.log.header
-rw-r--r-- 1 cassandra cassandra 112M 2011-06-24 20:54
CommitLog-1308947888397.log
-rw-r--r-- 1 cassandra cassandra 44 2011-06-24 20:47
CommitLog-1308947888397.log.header
{noformat}
The user KS has 2 CF's with 60 minute flush times. System KS had the default
settings which is 24 hours. Will create another ticket see if these can be
reduced or if it's something users should do, in this case it would not have
mattered.
I grabbed the log headers and used the tool in CASSANDRA-2828 and most of the
segments had the system CF's marked as dirty.
{noformat}
$ bin/logtool dirty /tmp/logs/commitlog/
Not connected to a server, Keyspace and Column Family names are not available.
/tmp/logs/commitlog/CommitLog-1308876643288.log.header
Keyspace Unknown:
Cf id 0: 444
/tmp/logs/commitlog/CommitLog-1308877711517.log.header
Keyspace Unknown:
Cf id 1: 68848763
...
/tmp/logs/commitlog/CommitLog-1308944451460.log.header
Keyspace Unknown:
Cf id 1: 61074
/tmp/logs/commitlog/CommitLog-1308945597471.log.header
Keyspace Unknown:
Cf id 1000: 43175492
Cf id 1: 108483
/tmp/logs/commitlog/CommitLog-1308946745380.log.header
Keyspace Unknown:
Cf id 1000: 239223
Cf id 1: 172211
/tmp/logs/commitlog/CommitLog-1308947888397.log.header
Keyspace Unknown:
Cf id 1001: 57595560
Cf id 1: 816960
Cf id 1000: 0
{noformat}
CF 0 is the Status / LocationInfo CF and 1 is the HintedHandof CF. I dont
have it now, but IIRC CFStats showed the LocationInfo CF with dirty ops.
I was able to repo a case where flushing the CF's did not mark the log
segments as obsolete (attached unit-test patch). Steps are:
1. Write to cf1 and flush.
2. Current log segment is marked as dirty at the CL position when the flush
started, CommitLog.discardCompletedSegmentsInternal()
3. Do not write to cf1 again.
4. Roll the log, my test does this manually.
5. Write to CF2 and flush.
6. Only CF2 is flushed because it is the only dirty CF.
cfs.maybeSwitchMemtable() is not called for cf1 and so log segment 1 is still
marked as dirty from cf1.
Step 5 is not essential, just matched what I thought was happening. I thought
SystemTable.updateToken() was called which does not flush, and this was the
last thing that happened.
The expired memtable thread created by Table uses the same cfs.forceFlush()
which is a no-op if the cf or it's secondary indexes are clean.
I think the same problem would exist in 0.8.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
