[jira] [Updated] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment
[ https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vijay updated CASSANDRA-3623: - Attachment: 0001-MMaped-Compression-segmented-file.patch Attached allows mmaped io on compressed SST's. We basically ignore the boundaries and split the files based on the chunks. use MMapedBuffer in CompressedSegmentedFile.getSegment -- Key: CASSANDRA-3623 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Fix For: 1.1 Attachments: 0001-MMaped-Compression-segmented-file.patch CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to use the MMap and hence a higher CPU on the nodes and higher latencies on reads. This ticket is to implement the TODO mentioned in CompressedRandomAccessReader // TODO refactor this to separate concept of buffer to avoid lots of read() syscalls and compression buffer but i think a separate class for the Buffer will be better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3611) Make checksum on a compressed blocks optional
[ https://issues.apache.org/jira/browse/CASSANDRA-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vijay updated CASSANDRA-3611: - Attachment: 0001-crc-check-chance.patch Make checksum on a compressed blocks optional - Key: CASSANDRA-3611 URL: https://issues.apache.org/jira/browse/CASSANDRA-3611 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 1.1 Attachments: 0001-crc-check-chance.patch Currently every uncompressed block is run against checksum algo, there is cpu overhead in doing same... We might want to make it configurable/optional for some use cases which might not require checksum all the time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3611) Make checksum on a compressed blocks optional
[ https://issues.apache.org/jira/browse/CASSANDRA-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vijay updated CASSANDRA-3611: - Attachment: (was: 0001-crc-check-chance.patch) Make checksum on a compressed blocks optional - Key: CASSANDRA-3611 URL: https://issues.apache.org/jira/browse/CASSANDRA-3611 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 1.1 Currently every uncompressed block is run against checksum algo, there is cpu overhead in doing same... We might want to make it configurable/optional for some use cases which might not require checksum all the time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3611) Make checksum on a compressed blocks optional
[ https://issues.apache.org/jira/browse/CASSANDRA-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vijay updated CASSANDRA-3611: - Attachment: 0001-crc-check-chance.patch Make checksum on a compressed blocks optional - Key: CASSANDRA-3611 URL: https://issues.apache.org/jira/browse/CASSANDRA-3611 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 1.1 Attachments: 0001-crc-check-chance.patch Currently every uncompressed block is run against checksum algo, there is cpu overhead in doing same... We might want to make it configurable/optional for some use cases which might not require checksum all the time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3611) Make checksum on a compressed blocks optional
[ https://issues.apache.org/jira/browse/CASSANDRA-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vijay updated CASSANDRA-3611: - Attachment: 0001-crc-check-chance.patch Make checksum on a compressed blocks optional - Key: CASSANDRA-3611 URL: https://issues.apache.org/jira/browse/CASSANDRA-3611 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 1.1 Attachments: 0001-crc-check-chance.patch Currently every uncompressed block is run against checksum algo, there is cpu overhead in doing same... We might want to make it configurable/optional for some use cases which might not require checksum all the time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3611) Make checksum on a compressed blocks optional
[ https://issues.apache.org/jira/browse/CASSANDRA-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vijay updated CASSANDRA-3611: - Attachment: (was: 0001-crc-check-chance.patch) Make checksum on a compressed blocks optional - Key: CASSANDRA-3611 URL: https://issues.apache.org/jira/browse/CASSANDRA-3611 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 1.1 Attachments: 0001-crc-check-chance.patch Currently every uncompressed block is run against checksum algo, there is cpu overhead in doing same... We might want to make it configurable/optional for some use cases which might not require checksum all the time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1600) Merge get_indexed_slices with get_range_slices
[
https://issues.apache.org/jira/browse/CASSANDRA-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-1600:
Attachment: 0004-Update-cql-to-not-use-deprecated-index-scan-v3.patch
0003-Allow-get_range_slices-to-apply-filter-to-a-sequenti-v3.patch
0002-thrift-generated-code-changes-v3.patch
0001-Add-optional-FilterClause-to-KeyRange-v3.patch
Attaching rebased version (the so-called v3 patchset).
For the most part this is a rebase of the preceding patches to trunk. But I've
also slightly modified the AbstractScanIterator idea of the preceding patches
to split it into the two classes of the new patch ExtendedFilter and
AbstractScanIterator. It felt like a better separation of concern for the
current trunk.
The last patch is new and just change CQL to use getRangeSlice with a filter
rather than the old index scan. That last part is not very tested except for
doing a stress test with '-o INDEXED_RANGE_SLICE' (and -L to enable CQL
obviously).
All the unit tests that are not already broken in trunk pass with this patch.
For the system tests (test/system/test_thrift_server.py), I got some weird
thrift error:
{noformat}
==
ERROR: Test that column ttled expires from KEYS index
--
Traceback (most recent call last):
File /usr/lib/pymodules/python2.7/nose/case.py, line 187, in runTest
self.test(*self.arg)
File /home/mcmanus/Git/cassandra/test/system/test_thrift_server.py, line
1911, in test_index_scan_expiring
result = get_range_slice(client, cp, sp, '', '', ConsistencyLevel.ONE,
clause)
File /home/mcmanus/Git/cassandra/test/system/test_thrift_server.py, line
218, in get_range_slice
return client.get_range_slices(parent, predicate, kr, cl)
File
/home/mcmanus/Git/cassandra/interface/thrift/gen-py/cassandra/Cassandra.py,
line 669, in get_range_slices
self.send_get_range_slices(column_parent, predicate, range,
consistency_level)
File
/home/mcmanus/Git/cassandra/interface/thrift/gen-py/cassandra/Cassandra.py,
line 679, in send_get_range_slices
args.write(self._oprot)
File
/home/mcmanus/Git/cassandra/interface/thrift/gen-py/cassandra/Cassandra.py,
line 3619, in write
oprot.writeI32(self.consistency_level)
File /usr/lib/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py,
line 110, in writeI32
buff = pack(!i, i32)
AttributeError: FilterClause instance has no attribute '__trunc__'
{noformat}
After that one, thrift is in a bad state and the next test throw another
completely wacko thrift exception during the setup:
{noformat}
==
ERROR: system.test_thrift_server.TestMutations.test_index_scan_uuid_names
--
Traceback (most recent call last):
File /usr/lib/pymodules/python2.7/nose/case.py, line 371, in setUp
try_run(self.inst, ('setup', 'setUp'))
File /usr/lib/pymodules/python2.7/nose/util.py, line 478, in try_run
return func()
File /home/mcmanus/Git/cassandra/test/system/__init__.py, line 113, in setUp
self.define_schema()
File /home/mcmanus/Git/cassandra/test/system/__init__.py, line 180, in
define_schema
self.client.system_add_keyspace(ks)
File
/home/mcmanus/Git/cassandra/interface/thrift/gen-py/cassandra/Cassandra.py,
line 1373, in system_add_keyspace
return self.recv_system_add_keyspace()
File
/home/mcmanus/Git/cassandra/interface/thrift/gen-py/cassandra/Cassandra.py,
line 1389, in recv_system_add_keyspace
raise x
TApplicationException: Required field 'consistency_level' was not present!
Struct:
get_range_slices_args(column_parent:ColumnParent(column_family:Indexed1),
predicate:SlicePredicate(slice_range:SliceRange(start:80 01 00 01 00 00 00 10
67 65 74 5F 72 61 6E 67 65 5F 73 6C 69 63 65 73 00 00 00 00 0C 00 01 0B 00 03
00 00 00 08 49 6E 64 65 78 65 64 31 00 0C 00 02 0C 00 02 0B 00 01 00 00 00 00,
finish:80 01 00 01 00 00 00 10 67 65 74 5F 72 61 6E 67 65 5F 73 6C 69 63 65 73
00 00 00 00 0C 00 01 0B 00 03 00 00 00 08 49 6E 64 65 78 65 64 31 00 0C 00 02
0C 00 02 0B 00 01 00 00 00 00 0B 00 02 00 00 00 00, reversed:false,
count:100)), range:KeyRange(start_key:80 01 00 01 00 00 00 10 67 65 74 5F 72 61
6E 67 65 5F 73 6C 69 63 65 73 00 00 00 00 0C 00 01 0B 00 03 00 00 00 08 49 6E
64 65 78 65 64 31 00 0C 00 02 0C 00 02 0B 00 01 00 00 00 00 0B 00 02 00 00 00
00 02 00 03 00 08 00 04 00 00 00 64 00 00 0C 00 03 0B 00 01 00 00 00 00,
end_key:80 01 00 01 00 00 00 10 67 65 74 5F 72 61 6E 67 65 5F 73 6C 69 63 65 73
00 00 00 00 0C 00 01 0B 00 03 00 00 00 08 49 6E 64 65 78 65 64 31 00 0C 00 02
0C 00 02 0B 00 01 00 00 00 00 0B 00 02 00 00 00 00 02 00 03 00 08 00 04 00 00
00 64 00 00 0C 00 03
[jira] [Updated] (CASSANDRA-1391) Allow Concurrent Schema Migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1391: --- Attachment: (was: 0003-oldVersion-removed-nit-fixed.patch) Allow Concurrent Schema Migrations -- Key: CASSANDRA-1391 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.0 Reporter: Stu Hood Assignee: Pavel Yaskevich Fix For: 1.1 Attachments: 0001-new-migration-schema-and-avro-methods-cleanup.patch, 0002-avro-removal.patch, CASSANDRA-1391.patch CASSANDRA-1292 fixed multiple migrations started from the same node to properly queue themselves, but it is still possible for migrations initiated on different nodes to conflict and leave the cluster in a bad state. Since the system_add/drop/rename methods are accessible directly from the client API, they should be completely safe for concurrent use. It should be possible to allow for most types of concurrent migrations by converting the UUID schema ID into a VersionVectorClock (as provided by CASSANDRA-580). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3658) Fix smallish problems find by FindBugs
[ https://issues.apache.org/jira/browse/CASSANDRA-3658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-3658: Attachment: 0010-Add-equals-and-hashCode-to-Expiring-column.patch 0009-remove-useless-assert-that-is-always-true.patch 0008-stop-pretending-Token-is-Serializable-LocalToken-is-.patch 0007-Inverse-argument-of-compare-instead-of-negating-to-a.patch 0006-Add-hashCode-method-when-equals-is-overriden.patch 0005-Protect-against-signed-byte-extension.patch 0004-Remove-dead-code.patch 0003-use-static-inner-class-when-possible.patch 0002-Avoid-race-when-reloading-snitch-file.patch 0001-Respect-Future-semantic.patch Fix smallish problems find by FindBugs -- Key: CASSANDRA-3658 URL: https://issues.apache.org/jira/browse/CASSANDRA-3658 Project: Cassandra Issue Type: Bug Components: Core Reporter: Sylvain Lebresne Priority: Minor Labels: fingbugs Fix For: 1.1 Attachments: 0001-Respect-Future-semantic.patch, 0002-Avoid-race-when-reloading-snitch-file.patch, 0003-use-static-inner-class-when-possible.patch, 0004-Remove-dead-code.patch, 0005-Protect-against-signed-byte-extension.patch, 0006-Add-hashCode-method-when-equals-is-overriden.patch, 0007-Inverse-argument-of-compare-instead-of-negating-to-a.patch, 0008-stop-pretending-Token-is-Serializable-LocalToken-is-.patch, 0009-remove-useless-assert-that-is-always-true.patch, 0010-Add-equals-and-hashCode-to-Expiring-column.patch I've just run (the newly released) FindBugs 2 out of curiosity. Attaching a number of patches related to issue raised by it. There is nothing major at all so all patches are against trunk. I've tried keep each issue to it's own patch with a self describing title. It far from covers all FindBugs alerts, but it's a picky tool so I've tried to address only what felt at least vaguely useful. Those are still mostly nits (only patch 2 is probably an actual bug). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3658) Fix smallish problems find by FindBugs
Fix smallish problems find by FindBugs -- Key: CASSANDRA-3658 URL: https://issues.apache.org/jira/browse/CASSANDRA-3658 Project: Cassandra Issue Type: Bug Components: Core Reporter: Sylvain Lebresne Priority: Minor Fix For: 1.1 Attachments: 0001-Respect-Future-semantic.patch, 0002-Avoid-race-when-reloading-snitch-file.patch, 0003-use-static-inner-class-when-possible.patch, 0004-Remove-dead-code.patch, 0005-Protect-against-signed-byte-extension.patch, 0006-Add-hashCode-method-when-equals-is-overriden.patch, 0007-Inverse-argument-of-compare-instead-of-negating-to-a.patch, 0008-stop-pretending-Token-is-Serializable-LocalToken-is-.patch, 0009-remove-useless-assert-that-is-always-true.patch, 0010-Add-equals-and-hashCode-to-Expiring-column.patch I've just run (the newly released) FindBugs 2 out of curiosity. Attaching a number of patches related to issue raised by it. There is nothing major at all so all patches are against trunk. I've tried keep each issue to it's own patch with a self describing title. It far from covers all FindBugs alerts, but it's a picky tool so I've tried to address only what felt at least vaguely useful. Those are still mostly nits (only patch 2 is probably an actual bug). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174772#comment-13174772 ] Sylvain Lebresne commented on CASSANDRA-3143: - This needs to be rebased. I really wouldn't mind if you were to squash all patches into just one btw. Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Attachments: 0001-global-key-cache.patch, 0002-global-row-cache-and-ASC.readSaved-changed-to-abstra.patch, 0003-CacheServiceMBean-and-correct-key-cache-loading.patch, 0004-key-row-cache-tests-and-tweaks.patch, 0005-cleanup-of-the-CFMetaData-and-thrift-avro-CfDef-and-.patch, 0006-row-key-cache-improvements-according-to-Sylvain-s-co.patch, 0007-caches-made-backward-compatible-second-round-of-chan.patch, 0008-row-key-cache-to-use-raw-key-instead-of-decorated.patch, 0009-3rd-Sylvain-comment-changes.patch Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: (was: 0001-global-key-cache.patch) Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: (was: 0002-global-row-cache-and-ASC.readSaved-changed-to-abstra.patch) Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: (was: 0004-key-row-cache-tests-and-tweaks.patch) Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: (was: 0005-cleanup-of-the-CFMetaData-and-thrift-avro-CfDef-and-.patch) Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: (was: 0003-CacheServiceMBean-and-correct-key-cache-loading.patch) Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: (was: 0007-caches-made-backward-compatible-second-round-of-chan.patch) Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: (was: 0006-row-key-cache-improvements-according-to-Sylvain-s-co.patch) Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: (was: 0008-row-key-cache-to-use-raw-key-instead-of-decorated.patch) Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: (was: 0009-3rd-Sylvain-comment-changes.patch) Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: CASSANDRA-3143-squashed.patch rebased and squashed. Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Attachments: CASSANDRA-3143-squashed.patch Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: (was: CASSANDRA-3143-squashed.patch) Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Attachments: CASSANDRA-3143-squashed.patch Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: CASSANDRA-3143-squashed.patch Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Attachments: CASSANDRA-3143-squashed.patch Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: (was: CASSANDRA-3143-squashed.patch) Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: CASSANDRA-3143-squashed.patch squashed commit properly rebased with trunk. Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Attachments: CASSANDRA-3143-squashed.patch Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3197) Separate input and output connection details in ConfigHelper
[ https://issues.apache.org/jira/browse/CASSANDRA-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mck SembWever updated CASSANDRA-3197: - Attachment: (was: CASSANDRA-3197.patch) Separate input and output connection details in ConfigHelper Key: CASSANDRA-3197 URL: https://issues.apache.org/jira/browse/CASSANDRA-3197 Project: Cassandra Issue Type: Improvement Components: Hadoop Affects Versions: 0.7.0 Reporter: Mck SembWever Assignee: Mck SembWever Priority: Minor Fix For: 1.1 Currently ConfigHelper's getInitialAddress(..) getRpcPort(..) and getPartitioner(..) presume CFIF will be using the same cluster as CFOF. The latter two are a problem for me as on the same servers i'm running two clusters, one w/ ByteOrderingPartitioner and the other with RP), and i would like to read from the BOP cluster and write to the RP cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3197) Separate input and output connection details in ConfigHelper
[ https://issues.apache.org/jira/browse/CASSANDRA-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mck SembWever updated CASSANDRA-3197: - Attachment: CASSANDRA-3197.patch Next patch attempt. Provides separate env vars for PIG's input/output. The existing env vars are still in use and used as fallback for both input and output. Separate input and output connection details in ConfigHelper Key: CASSANDRA-3197 URL: https://issues.apache.org/jira/browse/CASSANDRA-3197 Project: Cassandra Issue Type: Improvement Components: Hadoop Affects Versions: 0.7.0 Reporter: Mck SembWever Assignee: Mck SembWever Priority: Minor Fix For: 1.1 Attachments: CASSANDRA-3197.patch Currently ConfigHelper's getInitialAddress(..) getRpcPort(..) and getPartitioner(..) presume CFIF will be using the same cluster as CFOF. The latter two are a problem for me as on the same servers i'm running two clusters, one w/ ByteOrderingPartitioner and the other with RP), and i would like to read from the BOP cluster and write to the RP cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Trivial Update of MemtableSSTable by Bruno Leonardo Gonçalves
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The MemtableSSTable page has been changed by Bruno Leonardo Gonçalves: http://wiki.apache.org/cassandra/MemtableSSTable?action=diffrev1=18rev2=19 Comment: Updated link for Google's Bigtable paper ColumnFamilyStoreMBean exposes sstable space used as getLiveDiskSpaceUsed (only includes size of non-obsolete files) and getTotalDiskSpaceUsed (includes everything). == Further reading == - (The high-level memtable/sstable design as well as the Memtable and SSTable names come from Cassandra's sections 5.3 and 5.4 of [[http://labs.google.com/papers/bigtable.html|Google's Bigtable paper]], although some of the terminology around compaction differs.) + (The high-level memtable/sstable design as well as the Memtable and SSTable names come from Cassandra's sections 5.3 and 5.4 of [[http://research.google.com/archive/bigtable.html|Google's Bigtable paper]], although some of the terminology around compaction differs.) http://wiki.apache.org/cassandra/ArchitectureSSTable
[jira] [Created] (CASSANDRA-3659) Flush non-cfs backed secondary indexes along with CF
Flush non-cfs backed secondary indexes along with CF Key: CASSANDRA-3659 URL: https://issues.apache.org/jira/browse/CASSANDRA-3659 Project: Cassandra Issue Type: Bug Affects Versions: 1.0.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Priority: Minor Fix For: 1.0.7 Non CFS backed secondary indexes currently don't get flushed alongside CF. Only CFS backed ones do (i.e. KEYS) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3659) Flush non-cfs backed secondary indexes along with CF
[ https://issues.apache.org/jira/browse/CASSANDRA-3659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-3659: -- Attachment: v1-0001-CASSANDRA-3659-flush-non-cfs-backed-indexes.txt Flush non-cfs backed secondary indexes along with CF Key: CASSANDRA-3659 URL: https://issues.apache.org/jira/browse/CASSANDRA-3659 Project: Cassandra Issue Type: Bug Affects Versions: 1.0.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Priority: Minor Labels: secondary_index Fix For: 1.0.7 Attachments: v1-0001-CASSANDRA-3659-flush-non-cfs-backed-indexes.txt Non CFS backed secondary indexes currently don't get flushed alongside CF. Only CFS backed ones do (i.e. KEYS) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1600) Merge get_indexed_slices with get_range_slices
[
https://issues.apache.org/jira/browse/CASSANDRA-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174830#comment-13174830
]
Brandon Williams commented on CASSANDRA-1600:
-
bq. I'm at a loss on what is causing this. I double checked my thrift compiler
version (even recompiling it from scratch with no more success). I could use
someone looking to see if he gets the same error, or just look if he spot
something wrong in this test (I don't).
I get the exact same errors on multiple debian machines with thrift 0.6.0
Merge get_indexed_slices with get_range_slices
--
Key: CASSANDRA-1600
URL: https://issues.apache.org/jira/browse/CASSANDRA-1600
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Stu Hood
Assignee: Sylvain Lebresne
Fix For: 1.1
Attachments:
0001-Add-optional-FilterClause-to-KeyRange-and-support-do-v2.patch,
0001-Add-optional-FilterClause-to-KeyRange-and-support-doin.txt,
0001-Add-optional-FilterClause-to-KeyRange-v3.patch,
0002-allow-get_range_slices-to-apply-filter-to-a-sequenti-v2.patch,
0002-allow-get_range_slices-to-apply-filter-to-a-sequential.txt,
0002-thrift-generated-code-changes-v3.patch,
0003-Allow-get_range_slices-to-apply-filter-to-a-sequenti-v3.patch,
0004-Update-cql-to-not-use-deprecated-index-scan-v3.patch
From a comment on 1157:
{quote}
IndexClause only has a start key for get_indexed_slices, but it would seem
that the reasoning behind using 'KeyRange' for get_range_slices applies there
as well, since if you know the range you care about in the primary index, you
don't want to continue scanning until you exhaust 'count' (or the cluster).
Since it would appear that get_indexed_slices would benefit from a KeyRange,
why not smash get_(range|indexed)_slices together, and make IndexClause an
optional field on KeyRange?
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3659) Flush non-cfs backed secondary indexes along with CF
[ https://issues.apache.org/jira/browse/CASSANDRA-3659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174836#comment-13174836 ] Pavel Yaskevich commented on CASSANDRA-3659: +1 Flush non-cfs backed secondary indexes along with CF Key: CASSANDRA-3659 URL: https://issues.apache.org/jira/browse/CASSANDRA-3659 Project: Cassandra Issue Type: Bug Affects Versions: 1.0.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Priority: Minor Labels: secondary_index Fix For: 1.0.7 Attachments: v1-0001-CASSANDRA-3659-flush-non-cfs-backed-indexes.txt Non CFS backed secondary indexes currently don't get flushed alongside CF. Only CFS backed ones do (i.e. KEYS) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3155) Secondary index can report it's memory consumption
[ https://issues.apache.org/jira/browse/CASSANDRA-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-3155: -- Description: Non-CFS backed secondary indexes will consume RAM which should be reported back to Cassandra to be factored into it's flush by RAM amount. (was: A secondary index will consume RAM which should be reported back to Cassandra to be factored into it's flush by RAM amount.) Affects Version/s: 1.0.0 Fix Version/s: (was: 1.1) 1.0.7 Assignee: T Jake Luciani (was: Jason Rutherglen) Issue Type: Bug (was: Improvement) Secondary index can report it's memory consumption -- Key: CASSANDRA-3155 URL: https://issues.apache.org/jira/browse/CASSANDRA-3155 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Jason Rutherglen Assignee: T Jake Luciani Priority: Minor Fix For: 1.0.7 Non-CFS backed secondary indexes will consume RAM which should be reported back to Cassandra to be factored into it's flush by RAM amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment
[ https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3623: -- Reviewer: xedin Labels: compression (was: ) use MMapedBuffer in CompressedSegmentedFile.getSegment -- Key: CASSANDRA-3623 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Labels: compression Fix For: 1.1 Attachments: 0001-MMaped-Compression-segmented-file.patch CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to use the MMap and hence a higher CPU on the nodes and higher latencies on reads. This ticket is to implement the TODO mentioned in CompressedRandomAccessReader // TODO refactor this to separate concept of buffer to avoid lots of read() syscalls and compression buffer but i think a separate class for the Buffer will be better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3611) Make checksum on a compressed blocks optional
[ https://issues.apache.org/jira/browse/CASSANDRA-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3611: -- Reviewer: xedin Labels: compression (was: ) Make checksum on a compressed blocks optional - Key: CASSANDRA-3611 URL: https://issues.apache.org/jira/browse/CASSANDRA-3611 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Priority: Minor Labels: compression Fix For: 1.1 Attachments: 0001-crc-check-chance.patch Currently every uncompressed block is run against checksum algo, there is cpu overhead in doing same... We might want to make it configurable/optional for some use cases which might not require checksum all the time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3651) Truncate shouldn't rethrow timeouts as UA
[ https://issues.apache.org/jira/browse/CASSANDRA-3651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174858#comment-13174858 ] Jonathan Ellis edited comment on CASSANDRA-3651 at 12/22/11 3:54 PM: - Doesn't updating the Thrift signature mean (Java) clients now have to catch or rethrow TOE where before they did not? If so we should probably do this in 1.1. was (Author: jbellis): Doesn't updating the Thrift signature mean clients now have to catch or rethrow TOE where before they did not? If so we should probably do this in 1.1. Truncate shouldn't rethrow timeouts as UA - Key: CASSANDRA-3651 URL: https://issues.apache.org/jira/browse/CASSANDRA-3651 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.0.7 Attachments: 0001-Update-thrift-definition.txt, 0002-truncate-throws-TOE-on-timeout.txt Truncate is a very easy operation to timeout, but the timeouts rethrow as UnavailableException which is somewhat confusing. Instead it should throw TimedOutException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3651) Truncate shouldn't rethrow timeouts as UA
[ https://issues.apache.org/jira/browse/CASSANDRA-3651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174858#comment-13174858 ] Jonathan Ellis commented on CASSANDRA-3651: --- Doesn't updating the Thrift signature mean clients now have to catch or rethrow TOE where before they did not? If so we should probably do this in 1.1. Truncate shouldn't rethrow timeouts as UA - Key: CASSANDRA-3651 URL: https://issues.apache.org/jira/browse/CASSANDRA-3651 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.0.7 Attachments: 0001-Update-thrift-definition.txt, 0002-truncate-throws-TOE-on-timeout.txt Truncate is a very easy operation to timeout, but the timeouts rethrow as UnavailableException which is somewhat confusing. Instead it should throw TimedOutException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3660) Change syntax of cli for creating column families to be more intuitive
Change syntax of cli for creating column families to be more intuitive -- Key: CASSANDRA-3660 URL: https://issues.apache.org/jira/browse/CASSANDRA-3660 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Donald Smith Priority: Minor Currently, the syntax for creating column families is like this: create column family Users with comparator=UTF8Type and default_validation_class=UTF8Type and key_validation_class=UTF8Type; It's not clear what comparator and default_validation_class refer to. Much clearer would be: create column family Users with column_name_comparator=UTF8Type and column_value_validation_class=UTF8Type and key_validation_class=UTF8Type; BTW, instead of column_name_comparator, I'd actually prefer column_key_comparator since it seems more accurate to call column names column keys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3660) Change syntax of cli for creating column families to be more intuitive
[ https://issues.apache.org/jira/browse/CASSANDRA-3660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174863#comment-13174863 ] Donald Smith commented on CASSANDRA-3660: - On the cassandra users email list: Stephen Pope [stephen.p...@quest.com] wrote, I’d like to second this. I’ve been working with Cassandra for a good while now, but when I first started little things like this were confusing. R. Verlangen [ro...@us2.nl] wrote, I’d like to second this. I’ve been working with Cassandra for a good while now, but when I first started little things like this were confusing. Change syntax of cli for creating column families to be more intuitive -- Key: CASSANDRA-3660 URL: https://issues.apache.org/jira/browse/CASSANDRA-3660 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Donald Smith Priority: Minor Currently, the syntax for creating column families is like this: create column family Users with comparator=UTF8Type and default_validation_class=UTF8Type and key_validation_class=UTF8Type; It's not clear what comparator and default_validation_class refer to. Much clearer would be: create column family Users with column_name_comparator=UTF8Type and column_value_validation_class=UTF8Type and key_validation_class=UTF8Type; BTW, instead of column_name_comparator, I'd actually prefer column_key_comparator since it seems more accurate to call column names column keys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3651) Truncate shouldn't rethrow timeouts as UA
[ https://issues.apache.org/jira/browse/CASSANDRA-3651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174866#comment-13174866 ] Brandon Williams commented on CASSANDRA-3651: - Yes. :( Checked exceptions strike again. Truncate shouldn't rethrow timeouts as UA - Key: CASSANDRA-3651 URL: https://issues.apache.org/jira/browse/CASSANDRA-3651 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1 Attachments: 0001-Update-thrift-definition.txt, 0002-truncate-throws-TOE-on-timeout.txt Truncate is a very easy operation to timeout, but the timeouts rethrow as UnavailableException which is somewhat confusing. Instead it should throw TimedOutException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3651) Truncate shouldn't rethrow timeouts as UA
[ https://issues.apache.org/jira/browse/CASSANDRA-3651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3651: Fix Version/s: (was: 1.0.7) 1.1 Truncate shouldn't rethrow timeouts as UA - Key: CASSANDRA-3651 URL: https://issues.apache.org/jira/browse/CASSANDRA-3651 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1 Attachments: 0001-Update-thrift-definition.txt, 0002-truncate-throws-TOE-on-timeout.txt Truncate is a very easy operation to timeout, but the timeouts rethrow as UnavailableException which is somewhat confusing. Instead it should throw TimedOutException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3660) Change syntax of cli for creating column families to be more intuitive
[ https://issues.apache.org/jira/browse/CASSANDRA-3660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-3660. --- Resolution: Won't Fix cli is kept around for backwards compatiblity at this point; cqlsh is the future. Change syntax of cli for creating column families to be more intuitive -- Key: CASSANDRA-3660 URL: https://issues.apache.org/jira/browse/CASSANDRA-3660 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Donald Smith Priority: Minor Currently, the syntax for creating column families is like this: create column family Users with comparator=UTF8Type and default_validation_class=UTF8Type and key_validation_class=UTF8Type; It's not clear what comparator and default_validation_class refer to. Much clearer would be: create column family Users with column_name_comparator=UTF8Type and column_value_validation_class=UTF8Type and key_validation_class=UTF8Type; BTW, instead of column_name_comparator, I'd actually prefer column_key_comparator since it seems more accurate to call column names column keys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3651) Truncate shouldn't rethrow timeouts as UA
[ https://issues.apache.org/jira/browse/CASSANDRA-3651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3651: Attachment: (was: 0001-Update-thrift-definition.txt) Truncate shouldn't rethrow timeouts as UA - Key: CASSANDRA-3651 URL: https://issues.apache.org/jira/browse/CASSANDRA-3651 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1 Attachments: 0001-Update-thrift-definition.txt, 0002-truncate-throws-TOE-on-timeout.txt Truncate is a very easy operation to timeout, but the timeouts rethrow as UnavailableException which is somewhat confusing. Instead it should throw TimedOutException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3651) Truncate shouldn't rethrow timeouts as UA
[ https://issues.apache.org/jira/browse/CASSANDRA-3651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3651: Attachment: 0001-Update-thrift-definition.txt Truncate shouldn't rethrow timeouts as UA - Key: CASSANDRA-3651 URL: https://issues.apache.org/jira/browse/CASSANDRA-3651 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1 Attachments: 0001-Update-thrift-definition.txt, 0002-truncate-throws-TOE-on-timeout.txt Truncate is a very easy operation to timeout, but the timeouts rethrow as UnavailableException which is somewhat confusing. Instead it should throw TimedOutException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1222317 - in /cassandra/branches/cassandra-1.0: CHANGES.txt src/java/org/apache/cassandra/db/ColumnFamilyStore.java src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java
Author: jake
Date: Thu Dec 22 16:14:12 2011
New Revision: 1222317
URL: http://svn.apache.org/viewvc?rev=1222317view=rev
Log:
Flush non-cfs backed secondary indexes along with memtables
Patch by tjake; reviewed by xedin for: CASSANDRA-3659
Modified:
cassandra/branches/cassandra-1.0/CHANGES.txt
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java
Modified: cassandra/branches/cassandra-1.0/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/CHANGES.txt?rev=1222317r1=1222316r2=1222317view=diff
==
--- cassandra/branches/cassandra-1.0/CHANGES.txt (original)
+++ cassandra/branches/cassandra-1.0/CHANGES.txt Thu Dec 22 16:14:12 2011
@@ -7,6 +7,7 @@
(CASSANDRA-3335)
Merged from 0.8:
* prevent new nodes from thinking down nodes are up forever (CASSANDRA-3626)
+ * Flush non-cfs backed secondary indexes (CASSANDRA-3659)
1.0.6
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1222317r1=1222316r2=1222317view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Thu Dec 22 16:14:12 2011
@@ -49,6 +49,7 @@ import org.apache.cassandra.db.compactio
import org.apache.cassandra.db.filter.IFilter;
import org.apache.cassandra.db.filter.QueryFilter;
import org.apache.cassandra.db.filter.QueryPath;
+import org.apache.cassandra.db.index.SecondaryIndex;
import org.apache.cassandra.db.index.SecondaryIndexManager;
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.dht.*;
@@ -673,7 +674,7 @@ public class ColumnFamilyStore implement
logger.debug(flush position is {}, ctx);
// submit the memtable for any indexed sub-cfses, and our own.
-ListColumnFamilyStore icc = new ArrayListColumnFamilyStore();
+final ListColumnFamilyStore icc = new
ArrayListColumnFamilyStore();
// don't assume that this.memtable is dirty; forceFlush can bring
us here during index build even if it is not
for (ColumnFamilyStore cfs : concatWithIndexes())
{
@@ -705,6 +706,19 @@ public class ColumnFamilyStore implement
public void runMayThrow() throws InterruptedException,
IOException
{
latch.await();
+
+if (!icc.isEmpty())
+{
+//only valid when memtables exist
+
+for (SecondaryIndex index :
indexManager.getIndexesNotBackedByCfs())
+{
+// flush any non-cfs backed indexes
+logger.info(Flushing SecondaryIndex {}, index);
+index.forceBlockingFlush();
+}
+}
+
if (writeCommitLog)
{
// if we're not writing to the commit log, we are
replaying the log, so marking
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java?rev=1222317r1=1222316r2=1222317view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java
Thu Dec 22 16:14:12 2011
@@ -308,7 +308,27 @@ public class SecondaryIndexManager
return cfsList;
}
-
+/**
+ * @return all indexes which do *not* use a backing CFS internally
+ */
+public CollectionSecondaryIndex getIndexesNotBackedByCfs()
+{
+// we use identity map because per row indexes use same instance
+// across many columns
+IdentityHashMapSecondaryIndex, Object indexList = new
IdentityHashMapSecondaryIndex, Object();
+
+for (Map.EntryByteBuffer, SecondaryIndex entry :
indexesByColumn.entrySet())
+{
+ColumnFamilyStore cfs = entry.getValue().getIndexCfs();
+
+if (cfs == null)
+indexList.put(entry.getValue(), null);
+}
+
+return indexList.keySet();
+}
+
svn commit: r1222321 - in /cassandra/trunk: ./ contrib/ debian/ interface/thrift/gen-java/org/apache/cassandra/thrift/ pylib/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/index/
Author: jake Date: Thu Dec 22 16:16:18 2011 New Revision: 1222321 URL: http://svn.apache.org/viewvc?rev=1222321view=rev Log: merge from 1.0 Added: cassandra/trunk/debian/pyversions - copied unchanged from r1222319, cassandra/branches/cassandra-1.0/debian/pyversions cassandra/trunk/pylib/setup.py - copied unchanged from r1222319, cassandra/branches/cassandra-1.0/pylib/setup.py Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/debian/cassandra.install cassandra/trunk/debian/control cassandra/trunk/debian/rules cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java cassandra/trunk/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java cassandra/trunk/src/java/org/apache/cassandra/tools/NodeCmd.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 16:16:18 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 -/cassandra/branches/cassandra-1.0:1167085-1221481 +/cassandra/branches/cassandra-1.0:1167085-1222319 /cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/branches/cassandra-1.0.5:1208016 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1222321r1=1222320r2=1222321view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Thu Dec 22 16:16:18 2011 @@ -38,6 +38,7 @@ (CASSANDRA-3335) Merged from 0.8: * prevent new nodes from thinking down nodes are up forever (CASSANDRA-3626) + * Flush non-cfs backed secondary indexes (CASSANDRA-3659) 1.0.6 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 16:16:18 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 -/cassandra/branches/cassandra-1.0/contrib:1167085-1221481 +/cassandra/branches/cassandra-1.0/contrib:1167085-1222319 /cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/branches/cassandra-1.0.5/contrib:1208016 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Modified: cassandra/trunk/debian/cassandra.install URL: http://svn.apache.org/viewvc/cassandra/trunk/debian/cassandra.install?rev=1222321r1=1222320r2=1222321view=diff == --- cassandra/trunk/debian/cassandra.install (original) +++ cassandra/trunk/debian/cassandra.install Thu Dec 22 16:16:18 2011 @@ -11,5 +11,6 @@ bin/json2sstable usr/bin bin/sstable2json usr/bin bin/sstablekeys usr/bin bin/sstableloader usr/bin +bin/cqlsh usr/bin lib/*.jar usr/share/cassandra/lib lib/licenses usr/share/doc/cassandra Modified: cassandra/trunk/debian/control URL: http://svn.apache.org/viewvc/cassandra/trunk/debian/control?rev=1222321r1=1222320r2=1222321view=diff == --- cassandra/trunk/debian/control (original) +++ cassandra/trunk/debian/control Thu Dec 22 16:16:18 2011 @@ -2,7 +2,8 @@ Source: cassandra Section: misc Priority: extra Maintainer: Eric Evans eev...@apache.org -Build-Depends: debhelper (= 5), openjdk-6-jdk (= 6b11) | java6-sdk, ant (= 1.7), ant-optional (= 1.7), subversion +Uploaders: Sylvain Lebresne slebre...@apache.org +Build-Depends: debhelper (= 5), openjdk-6-jdk (= 6b11) | java6-sdk, ant (= 1.7), ant-optional (= 1.7), subversion, python-support (= 0.6) Homepage: http://cassandra.apache.org Vcs-Svn: https://svn.apache.org/repos/asf/cassandra/trunk Vcs-Browser: http://svn.apache.org/viewvc/cassandra/trunk @@
[jira] [Commented] (CASSANDRA-3651) Truncate shouldn't rethrow timeouts as UA
[ https://issues.apache.org/jira/browse/CASSANDRA-3651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174882#comment-13174882 ] Jonathan Ellis commented on CASSANDRA-3651: --- +1 Truncate shouldn't rethrow timeouts as UA - Key: CASSANDRA-3651 URL: https://issues.apache.org/jira/browse/CASSANDRA-3651 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1 Attachments: 0001-Update-thrift-definition.txt, 0002-truncate-throws-TOE-on-timeout.txt Truncate is a very easy operation to timeout, but the timeouts rethrow as UnavailableException which is somewhat confusing. Instead it should throw TimedOutException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3661) Re-introduce timeout debug messages in CassandraServer
Re-introduce timeout debug messages in CassandraServer
--
Key: CASSANDRA-3661
URL: https://issues.apache.org/jira/browse/CASSANDRA-3661
Project: Cassandra
Issue Type: Improvement
Components: API
Affects Versions: 0.8.0
Reporter: Jonathan Ellis
Assignee: Brandon Williams
Priority: Trivial
Fix For: 0.8.10, 1.0.7
In 0.7 we log at debug when returning TOE back to the client:
{code}
. catch (TimeoutException e)
{
logger.debug(... timed out);
throw new TimedOutException();
}
{code}
This is primarily useful when reading through debug logs to make it obvious
when StorageProxy gave up waiting and cleared its callbacks.
At some point this got removed from 0.8+. (Or possibly never got correctly
merged upwards.) Let's fix this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment
[ https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174886#comment-13174886 ] Pavel Yaskevich commented on CASSANDRA-3623: With read()/write() operations data gets copied 2 times - from disk to page cache and from page cache to user buffer, the benefit of the mmap is that it allows to skip the second copy by directly mapping file (in case of file mapping) contents to the process address space. But if you take a look at CompressedFileDataInput.reBuffer() you would see that that second copy is still made. I don't see how mmap'ed I/O gives us any real benefit operating on the compressed files. use MMapedBuffer in CompressedSegmentedFile.getSegment -- Key: CASSANDRA-3623 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Labels: compression Fix For: 1.1 Attachments: 0001-MMaped-Compression-segmented-file.patch CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to use the MMap and hence a higher CPU on the nodes and higher latencies on reads. This ticket is to implement the TODO mentioned in CompressedRandomAccessReader // TODO refactor this to separate concept of buffer to avoid lots of read() syscalls and compression buffer but i think a separate class for the Buffer will be better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3143) Global caches (key/row)
[
https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174887#comment-13174887
]
Sylvain Lebresne commented on CASSANDRA-3143:
-
We're getting there :)
A few more comments however:
* When saving the cache, the 'writeKeyQuietly' make it possible to write a
corrupted file (maybe only part of a key has been written) and has the risk of
flooding the log (if one key throw an IOException, chances are the next one on
this file will too). I think we'd have less problem just stop all saving if an
IOError occurs.
* In CFS.initRowCache(), the test {{ if (cachedRowsRead++
rowCache.getCapacity()) }} is now not correct since getCapacity is a memory
size. For now, I'm fine removing the test and say that 'if you reduce the size,
you may not get you hottest keys loaded on startup'. Though ultimately we'll
probably need to fix that.
On what we expose through MBean:
* I think I was preferring the old way of using the InstrumentingCacheMBean,
rather than to have lots of {get,set}RowCache, {get,set}KeyCache method that
forward to the InstrumentingCache ones. Basically I think it's more clear to
have Cache-RowCache-infos and Cache-KeyCache-infos, rather than
CacheServices-allInfos. It's also more easily extensible if we ever add some
new cache.
* In any case, we don't register the InstrumentingCacheMBean anymore and the
CacheService one doesn't expose the hit rate nor the number of requests
processed (and we can remove InstrumentingCacheMBean if we're not going to use
it).
* Both set{Key,Row}CacheSavePeriodInSeconds and saveCaches disregard the
{Key,Row}CacheKeysToSave setting, while they probably shouldn't.
* I'd rename getRowCacheCapacity() to getRwoCacheCapacityInMB() to match the
set method.
And a bunch of very minor nitpicks that I just happened to gather during the
review:
* In AutoSavingCache.saveCache, I'd log the Deleting old files message at
DEBUG.
* In DatabaseDescriptor, there's a wrongly placed import
* In CFS, we should probably remove the getKeyCache method, to emphasis it's
now a global thing. Same in DataTracker.
* The comment from {{ data.addSSTables(sstables); // this will call
updateCacheSizes() for us }} in CFS.loadNewSSTables is outdated
* DK.java uselessly import RowCacheKey (and only have a very gratuitous
codeStyle change btw).
* I would rename the CacheService MBean name to more simply
org.apache.cassandra.db:type=Caches.
I'll also note for posterity that by removing the DK from the cache keys, we're
trading off memory for cpu (since we have to redecorate for each use). Don't
get me wrong, it's likely a good trade-off, just wanted to write it down.
Global caches (key/row)
---
Key: CASSANDRA-3143
URL: https://issues.apache.org/jira/browse/CASSANDRA-3143
Project: Cassandra
Issue Type: Improvement
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
Priority: Minor
Labels: Core
Fix For: 1.1
Attachments: CASSANDRA-3143-squashed.patch
Caches are difficult to configure well as ColumnFamilies are added, similar
to how memtables were difficult pre-CASSANDRA-2006.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1222340 - in /cassandra/trunk: interface/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/thrift/
Author: brandonwilliams
Date: Thu Dec 22 16:41:21 2011
New Revision: 1222340
URL: http://svn.apache.org/viewvc?rev=1222340view=rev
Log:
Truncate throws TOE on timeout instead of UA.
Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-3651
Modified:
cassandra/trunk/interface/cassandra.thrift
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Constants.java
cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java
Modified: cassandra/trunk/interface/cassandra.thrift
URL:
http://svn.apache.org/viewvc/cassandra/trunk/interface/cassandra.thrift?rev=1222340r1=1222339r2=1222340view=diff
==
--- cassandra/trunk/interface/cassandra.thrift (original)
+++ cassandra/trunk/interface/cassandra.thrift Thu Dec 22 16:41:21 2011
@@ -46,7 +46,7 @@ namespace rb CassandraThrift
# for every edit that doesn't result in a change to major/minor.
#
# See the Semantic Versioning Specification (SemVer) http://semver.org.
-const string VERSION = 19.20.0
+const string VERSION = 19.22.1
#
@@ -599,7 +599,7 @@ service Cassandra {
some hosts are down.
*/
void truncate(1:required string cfname)
- throws (1: InvalidRequestException ire, 2: UnavailableException ue),
+ throws (1: InvalidRequestException ire, 2: UnavailableException ue, 3:
TimedOutException te),
Modified:
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java?rev=1222340r1=1222339r2=1222340view=diff
==
---
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
(original)
+++
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
Thu Dec 22 16:41:21 2011
@@ -186,7 +186,7 @@ public class Cassandra {
*
* @param cfname
*/
-public void truncate(String cfname) throws InvalidRequestException,
UnavailableException, org.apache.thrift.TException;
+public void truncate(String cfname) throws InvalidRequestException,
UnavailableException, TimedOutException, org.apache.thrift.TException;
/**
* for each schema version present in the cluster, returns a list of nodes
at that version.
@@ -1070,7 +1070,7 @@ public class Cassandra {
return;
}
-public void truncate(String cfname) throws InvalidRequestException,
UnavailableException, org.apache.thrift.TException
+public void truncate(String cfname) throws InvalidRequestException,
UnavailableException, TimedOutException, org.apache.thrift.TException
{
send_truncate(cfname);
recv_truncate();
@@ -1086,7 +1086,7 @@ public class Cassandra {
oprot_.getTransport().flush();
}
-public void recv_truncate() throws InvalidRequestException,
UnavailableException, org.apache.thrift.TException
+public void recv_truncate() throws InvalidRequestException,
UnavailableException, TimedOutException, org.apache.thrift.TException
{
org.apache.thrift.protocol.TMessage msg = iprot_.readMessageBegin();
if (msg.type == org.apache.thrift.protocol.TMessageType.EXCEPTION) {
@@ -1106,6 +1106,9 @@ public class Cassandra {
if (result.ue != null) {
throw result.ue;
}
+ if (result.te != null) {
+throw result.te;
+ }
return;
}
@@ -2422,7 +2425,7 @@ public class Cassandra {
prot.writeMessageEnd();
}
- public void getResult() throws InvalidRequestException,
UnavailableException, org.apache.thrift.TException {
+ public void getResult() throws InvalidRequestException,
UnavailableException, TimedOutException, org.apache.thrift.TException {
if (getState() !=
org.apache.thrift.async.TAsyncMethodCall.State.RESPONSE_READ) {
throw new IllegalStateException(Method call not finished!);
}
@@ -3683,6 +3686,8 @@ public class Cassandra {
result.ire = ire;
} catch (UnavailableException ue) {
result.ue = ue;
+} catch (TimedOutException te) {
+ result.te = te;
} catch (Throwable th) {
LOGGER.error(Internal error processing truncate, th);
org.apache.thrift.TApplicationException x = new
org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.INTERNAL_ERROR,
Internal error processing truncate);
@@ -9353,6 +9358,8 @@ public class Cassandra {
private void readObject(java.io.ObjectInputStream in) throws
java.io.IOException, ClassNotFoundException {
try {
+// it doesn't seem like you should have to do this, but java
serialization is wacky, and doesn't call the default constructor.
+
[jira] [Commented] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174893#comment-13174893 ] Jonathan Ellis commented on CASSANDRA-3143: --- bq. I'll also note for posterity that by removing the DK from the cache keys, we're trading off memory for cpu (since we have to redecorate for each use) Why is that? Shouldn't we be looking up by (cfid, raw key)? We already have the DK in the getPosition / getCachedPosition call tree, we shouldn't need to store it in the cache to avoid recomputing it. Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Attachments: CASSANDRA-3143-squashed.patch Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1222347 - /cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java
Author: brandonwilliams
Date: Thu Dec 22 16:56:17 2011
New Revision: 1222347
URL: http://svn.apache.org/viewvc?rev=1222347view=rev
Log:
Fix cli breakage from 3651
Modified:
cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java
Modified: cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java?rev=1222347r1=1222346r2=1222347view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java Thu Dec 22
16:56:17 2011
@@ -1431,7 +1431,7 @@ public class CliClient
}
// TRUNCATE columnFamily
-private void executeTruncate(String columnFamily) throws TException,
InvalidRequestException, UnavailableException
+private void executeTruncate(String columnFamily) throws TException,
InvalidRequestException, UnavailableException, TimedOutException
{
if (!CliMain.isConnected() || !hasKeySpace())
return;
[jira] [Commented] (CASSANDRA-3658) Fix smallish problems find by FindBugs
[ https://issues.apache.org/jira/browse/CASSANDRA-3658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174898#comment-13174898 ] Jonathan Ellis commented on CASSANDRA-3658: --- +1 on all but 09; I'd rather leave that assert alone since its point is to catch a bug if we change the signature of scanner Fix smallish problems find by FindBugs -- Key: CASSANDRA-3658 URL: https://issues.apache.org/jira/browse/CASSANDRA-3658 Project: Cassandra Issue Type: Bug Components: Core Reporter: Sylvain Lebresne Priority: Minor Labels: fingbugs Fix For: 1.1 Attachments: 0001-Respect-Future-semantic.patch, 0002-Avoid-race-when-reloading-snitch-file.patch, 0003-use-static-inner-class-when-possible.patch, 0004-Remove-dead-code.patch, 0005-Protect-against-signed-byte-extension.patch, 0006-Add-hashCode-method-when-equals-is-overriden.patch, 0007-Inverse-argument-of-compare-instead-of-negating-to-a.patch, 0008-stop-pretending-Token-is-Serializable-LocalToken-is-.patch, 0009-remove-useless-assert-that-is-always-true.patch, 0010-Add-equals-and-hashCode-to-Expiring-column.patch I've just run (the newly released) FindBugs 2 out of curiosity. Attaching a number of patches related to issue raised by it. There is nothing major at all so all patches are against trunk. I've tried keep each issue to it's own patch with a self describing title. It far from covers all FindBugs alerts, but it's a picky tool so I've tried to address only what felt at least vaguely useful. Those are still mostly nits (only patch 2 is probably an actual bug). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2893) Add row-level isolation
[ https://issues.apache.org/jira/browse/CASSANDRA-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174903#comment-13174903 ] Jonathan Ellis commented on CASSANDRA-2893: --- Okay, so more accurately about 6% on uncontended and 50% on contended. Since the contended case is likely to be where people want the isolation, that seems reasonable. Add row-level isolation --- Key: CASSANDRA-2893 URL: https://issues.apache.org/jira/browse/CASSANDRA-2893 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Sylvain Lebresne Priority: Minor Fix For: 1.1 Attachments: 0001-Move-deletion-infos-into-ISortedColumns-v2.patch, 0001-Move-deletion-infos-into-ISortedColumns.patch, 0002-Make-memtable-use-CF.addAll-v2.patch, 0002-Make-memtable-use-CF.addAll.patch, 0003-Add-AtomicSortedColumn-and-snapTree-v2.patch, 0003-Add-AtomicSortedColumn-and-snapTree.patch, latency-plain.svg, latency.svg, snaptree-0.1-SNAPSHOT.jar This could be done using an the atomic ConcurrentMap operations from the Memtable and something like http://code.google.com/p/pcollections/ to replace the ConcurrentSkipListMap in ThreadSafeSortedColumns. The trick is that pcollections does not provide a SortedMap, so we probably need to write our own. Googling [persistent sortedmap] I found http://code.google.com/p/actord/source/browse/trunk/actord/src/main/scala/ff/collection (in scala) and http://clojure.org/data_structures#Data Structures-Maps. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174904#comment-13174904 ] Sylvain Lebresne commented on CASSANDRA-3143: - Oups, you're right, I was looking of the case of storing a new row in the rowCache, where the patch does decorate the key, and got a bit carried away. Especially so since even if the case of storing a row in the cache we can easily avoid the recomputation, it's just an inefficiency of the patch (that should be fixed). Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Attachments: CASSANDRA-3143-squashed.patch Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2893) Add row-level isolation
[ https://issues.apache.org/jira/browse/CASSANDRA-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174907#comment-13174907 ] Brandon Williams commented on CASSANDRA-2893: - bq. Okay, so more accurately about 6% on uncontended and 50% on contended. Since the contended case is likely to be where people want the isolation, that seems reasonable. +1 (though it's also important to remember this is in a local context; network latency makes these percentages lower in practice) Add row-level isolation --- Key: CASSANDRA-2893 URL: https://issues.apache.org/jira/browse/CASSANDRA-2893 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Sylvain Lebresne Priority: Minor Fix For: 1.1 Attachments: 0001-Move-deletion-infos-into-ISortedColumns-v2.patch, 0001-Move-deletion-infos-into-ISortedColumns.patch, 0002-Make-memtable-use-CF.addAll-v2.patch, 0002-Make-memtable-use-CF.addAll.patch, 0003-Add-AtomicSortedColumn-and-snapTree-v2.patch, 0003-Add-AtomicSortedColumn-and-snapTree.patch, latency-plain.svg, latency.svg, snaptree-0.1-SNAPSHOT.jar This could be done using an the atomic ConcurrentMap operations from the Memtable and something like http://code.google.com/p/pcollections/ to replace the ConcurrentSkipListMap in ThreadSafeSortedColumns. The trick is that pcollections does not provide a SortedMap, so we probably need to write our own. Googling [persistent sortedmap] I found http://code.google.com/p/actord/source/browse/trunk/actord/src/main/scala/ff/collection (in scala) and http://clojure.org/data_structures#Data Structures-Maps. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2893) Add row-level isolation
[
https://issues.apache.org/jira/browse/CASSANDRA-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174911#comment-13174911
]
Jonathan Ellis commented on CASSANDRA-2893:
---
On the code side: should we just get rid of AbstractColumnContainer now since
it's basically just a wrapper around ISortedColumns?
Doesn't Guava Functions.identity do the same as ACC.identity?
{quote}
+ * In case we are adding a lot of columns, failing the final compare
+ * and swap could be expensive. To mitigate, we check we haven't been
+ * beaten by another thread after every column addition. If we have,
+ * we bail early, avoiding unnecessary work if possible.
{quote}
I wonder if this is premature optimization or even counterproductive for the
uncontended case. Did you do any testing around this?
More generally it may be worth looking at the before- and after- yourkit traces
for uncontended to see if there's any improvement we can make in
AtomicSortedColumns or if most of the overhead is from SnapTreeMap.
Add row-level isolation
---
Key: CASSANDRA-2893
URL: https://issues.apache.org/jira/browse/CASSANDRA-2893
Project: Cassandra
Issue Type: Improvement
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
Priority: Minor
Fix For: 1.1
Attachments: 0001-Move-deletion-infos-into-ISortedColumns-v2.patch,
0001-Move-deletion-infos-into-ISortedColumns.patch,
0002-Make-memtable-use-CF.addAll-v2.patch,
0002-Make-memtable-use-CF.addAll.patch,
0003-Add-AtomicSortedColumn-and-snapTree-v2.patch,
0003-Add-AtomicSortedColumn-and-snapTree.patch, latency-plain.svg,
latency.svg, snaptree-0.1-SNAPSHOT.jar
This could be done using an the atomic ConcurrentMap operations from the
Memtable and something like http://code.google.com/p/pcollections/ to replace
the ConcurrentSkipListMap in ThreadSafeSortedColumns. The trick is that
pcollections does not provide a SortedMap, so we probably need to write our
own.
Googling [persistent sortedmap] I found
http://code.google.com/p/actord/source/browse/trunk/actord/src/main/scala/ff/collection
(in scala) and http://clojure.org/data_structures#Data Structures-Maps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3143: --- Attachment: 0002-fixes.patch separate patch with fixes attached. Global caches (key/row) --- Key: CASSANDRA-3143 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 Project: Cassandra Issue Type: Improvement Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Labels: Core Fix For: 1.1 Attachments: 0002-fixes.patch, CASSANDRA-3143-squashed.patch Caches are difficult to configure well as ColumnFamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3651) Truncate shouldn't rethrow timeouts as UA
[ https://issues.apache.org/jira/browse/CASSANDRA-3651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174916#comment-13174916 ] Hudson commented on CASSANDRA-3651: --- Integrated in Cassandra #1264 (See [https://builds.apache.org/job/Cassandra/1264/]) Truncate throws TOE on timeout instead of UA. Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-3651 brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1222340 Files : * /cassandra/trunk/interface/cassandra.thrift * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java * /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Constants.java * /cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java Truncate shouldn't rethrow timeouts as UA - Key: CASSANDRA-3651 URL: https://issues.apache.org/jira/browse/CASSANDRA-3651 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1 Attachments: 0001-Update-thrift-definition.txt, 0002-truncate-throws-TOE-on-timeout.txt Truncate is a very easy operation to timeout, but the timeouts rethrow as UnavailableException which is somewhat confusing. Instead it should throw TimedOutException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1600) Merge get_indexed_slices with get_range_slices
[
https://issues.apache.org/jira/browse/CASSANDRA-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-1600:
Attachment: (was:
0003-Allow-get_range_slices-to-apply-filter-to-a-sequenti-v3.patch)
Merge get_indexed_slices with get_range_slices
--
Key: CASSANDRA-1600
URL: https://issues.apache.org/jira/browse/CASSANDRA-1600
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Stu Hood
Assignee: Sylvain Lebresne
Fix For: 1.1
Attachments:
0001-Add-optional-FilterClause-to-KeyRange-and-support-do-v2.patch,
0001-Add-optional-FilterClause-to-KeyRange-and-support-doin.txt,
0001-Add-optional-FilterClause-to-KeyRange-v3.patch,
0002-allow-get_range_slices-to-apply-filter-to-a-sequenti-v2.patch,
0002-allow-get_range_slices-to-apply-filter-to-a-sequential.txt,
0002-thrift-generated-code-changes-v3.patch,
0003-Allow-get_range_slices-to-apply-filter-to-a-sequenti-v3.patch,
0004-Update-cql-to-not-use-deprecated-index-scan-v3.patch
From a comment on 1157:
{quote}
IndexClause only has a start key for get_indexed_slices, but it would seem
that the reasoning behind using 'KeyRange' for get_range_slices applies there
as well, since if you know the range you care about in the primary index, you
don't want to continue scanning until you exhaust 'count' (or the cluster).
Since it would appear that get_indexed_slices would benefit from a KeyRange,
why not smash get_(range|indexed)_slices together, and make IndexClause an
optional field on KeyRange?
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3661) Re-introduce timeout debug messages in CassandraServer
[
https://issues.apache.org/jira/browse/CASSANDRA-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174921#comment-13174921
]
Jonathan Ellis commented on CASSANDRA-3661:
---
+1
Re-introduce timeout debug messages in CassandraServer
--
Key: CASSANDRA-3661
URL: https://issues.apache.org/jira/browse/CASSANDRA-3661
Project: Cassandra
Issue Type: Improvement
Components: API
Affects Versions: 0.8.0
Reporter: Jonathan Ellis
Assignee: Brandon Williams
Priority: Trivial
Labels: thrift
Fix For: 0.8.10, 1.0.7
Attachments: 3661.txt
In 0.7 we log at debug when returning TOE back to the client:
{code}
. catch (TimeoutException e)
{
logger.debug(... timed out);
throw new TimedOutException();
}
{code}
This is primarily useful when reading through debug logs to make it obvious
when StorageProxy gave up waiting and cleared its callbacks.
At some point this got removed from 0.8+. (Or possibly never got correctly
merged upwards.) Let's fix this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1600) Merge get_indexed_slices with get_range_slices
[
https://issues.apache.org/jira/browse/CASSANDRA-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-1600:
Attachment:
0003-Allow-get_range_slices-to-apply-filter-to-a-sequenti-v3.patch
Ok, turns out that was just the get_range_slice calls missing an argument. I've
just updated patch 3 with the tests fixed. Ready for review :)
Merge get_indexed_slices with get_range_slices
--
Key: CASSANDRA-1600
URL: https://issues.apache.org/jira/browse/CASSANDRA-1600
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Stu Hood
Assignee: Sylvain Lebresne
Fix For: 1.1
Attachments:
0001-Add-optional-FilterClause-to-KeyRange-and-support-do-v2.patch,
0001-Add-optional-FilterClause-to-KeyRange-and-support-doin.txt,
0001-Add-optional-FilterClause-to-KeyRange-v3.patch,
0002-allow-get_range_slices-to-apply-filter-to-a-sequenti-v2.patch,
0002-allow-get_range_slices-to-apply-filter-to-a-sequential.txt,
0002-thrift-generated-code-changes-v3.patch,
0003-Allow-get_range_slices-to-apply-filter-to-a-sequenti-v3.patch,
0004-Update-cql-to-not-use-deprecated-index-scan-v3.patch
From a comment on 1157:
{quote}
IndexClause only has a start key for get_indexed_slices, but it would seem
that the reasoning behind using 'KeyRange' for get_range_slices applies there
as well, since if you know the range you care about in the primary index, you
don't want to continue scanning until you exhaust 'count' (or the cluster).
Since it would appear that get_indexed_slices would benefit from a KeyRange,
why not smash get_(range|indexed)_slices together, and make IndexClause an
optional field on KeyRange?
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment
[ https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174924#comment-13174924 ] Vijay commented on CASSANDRA-3623: -- How about the system calls and its over head? http://stackoverflow.com/questions/5614206/buffered-randomaccessfile-java, the point i am trying to make is, if you consider the chunk as hot and you would access them often enough (instead of considering it as a columns). use MMapedBuffer in CompressedSegmentedFile.getSegment -- Key: CASSANDRA-3623 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Labels: compression Fix For: 1.1 Attachments: 0001-MMaped-Compression-segmented-file.patch CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to use the MMap and hence a higher CPU on the nodes and higher latencies on reads. This ticket is to implement the TODO mentioned in CompressedRandomAccessReader // TODO refactor this to separate concept of buffer to avoid lots of read() syscalls and compression buffer but i think a separate class for the Buffer will be better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1222372 - /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
Author: brandonwilliams
Date: Thu Dec 22 17:49:05 2011
New Revision: 1222372
URL: http://svn.apache.org/viewvc?rev=1222372view=rev
Log:
Re-introduce timeout debug messages in CassandraServer.
Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-3661
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java?rev=1222372r1=1222371r2=1222372view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
Thu Dec 22 17:49:05 2011
@@ -133,6 +133,7 @@ public class CassandraServer implements
}
catch (TimeoutException e)
{
+logger.debug(... timed out);
throw new TimedOutException();
}
catch (IOException e)
@@ -579,6 +580,7 @@ public class CassandraServer implements
}
catch (TimeoutException e)
{
+logger.debug(... timed out);
throw new TimedOutException();
}
}
@@ -642,6 +644,7 @@ public class CassandraServer implements
}
catch (TimeoutException e)
{
+logger.debug(... timed out);
throw new TimedOutException();
}
catch (IOException e)
@@ -688,6 +691,7 @@ public class CassandraServer implements
}
catch (TimeoutException e)
{
+logger.debug(... timed out);
throw new TimedOutException();
}
return thriftifyKeySlices(rows, column_parent, column_predicate);
@@ -1036,6 +1040,7 @@ public class CassandraServer implements
}
catch (TimeoutException e)
{
+logger.debug(... timed out);
throw (UnavailableException) new
UnavailableException().initCause(e);
}
catch (IOException e)
svn commit: r1222374 - in /cassandra/branches/cassandra-1.0: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/thrift/
Author: brandonwilliams Date: Thu Dec 22 17:53:42 2011 New Revision: 1222374 URL: http://svn.apache.org/viewvc?rev=1222374view=rev Log: Merge 3661 from 0.8 Modified: cassandra/branches/cassandra-1.0/ (props changed) cassandra/branches/cassandra-1.0/contrib/ (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/thrift/CassandraServer.java Propchange: cassandra/branches/cassandra-1.0/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 17:53:42 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7:1026516-1211709 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1212854,1212938,1214916,1220926 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1212854,1212938,1214916,1220926,1222372 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/branches/cassandra-1.0:1167106,1167185 Propchange: cassandra/branches/cassandra-1.0/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 17:53:42 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 /cassandra/branches/cassandra-0.7/contrib:1026516-1211709 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1212854,1212938,1214916,1220926 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1212854,1212938,1214916,1220926,1222372 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/branches/cassandra-1.0/contrib:1167106,1167185 Propchange: cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 17:53:42 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1211709 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1212854,1212938,1214916,1220926 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1212854,1212938,1214916,1220926,1222372 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018 /cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167106,1167185 Propchange: cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 17:53:42 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1211709 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090934-1125013,1125019-1212854,1212938,1214916,1220926 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090934-1125013,1125019-1212854,1212938,1214916,1220926,1222372
[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment
[ https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174944#comment-13174944 ] Pavel Yaskevich commented on CASSANDRA-3623: Cost of the syscall is negligable comparing cost of buffer copy and (in the worst case) I/O. use MMapedBuffer in CompressedSegmentedFile.getSegment -- Key: CASSANDRA-3623 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Labels: compression Fix For: 1.1 Attachments: 0001-MMaped-Compression-segmented-file.patch CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to use the MMap and hence a higher CPU on the nodes and higher latencies on reads. This ticket is to implement the TODO mentioned in CompressedRandomAccessReader // TODO refactor this to separate concept of buffer to avoid lots of read() syscalls and compression buffer but i think a separate class for the Buffer will be better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2893) Add row-level isolation
[ https://issues.apache.org/jira/browse/CASSANDRA-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174948#comment-13174948 ] Sylvain Lebresne commented on CASSANDRA-2893: - bq. should we just get rid of AbstractColumnContainer now since it's basically just a wrapper around ISortedColumns? It's still useful to factor the code between ColumnFamily and SuperColumn. If we remove it, we'll basically have to duplicate ACC into both CF and SC. But I'll be more than happy to remove it when/if we remove SCs :P bq. Doesn't Guava Functions.identity do the same as ACC.identity? Totally, I guess I don't know Guava too well. I'll replace it. bq. I wonder if this is premature optimization or even counterproductive for the uncontended case. Did you do any testing around this? To be honest, I did not test it. But the rational is that I would be surprised if checking an atomic ref get (even on each iteration) has any kind of visible impact (compared to adding a column to a snaptree), while I have no doubt that in the contended case with non trivially small batches not doing this will have a visible impact. That was more about trading a probably non-noticeable hit on non-contended to avoid any bad outliers. I also though about doing the the reference check only every few iterations, but I figured that would likely don't change anything since again an atomic ref get should be really fast. But obviously it doesn't cost much to check those assumptions and I'll do some testing. I'll also note that we could easily change snaptree by a persistent sorted map. This could be faster than snaptree because there would be any internal synchronization (while snaptree has internal synchronization, that we mostly don't use, except for the fact that they allow a concurrently safe copy-on-write). I have a persistent store map implementation almost ready so I'll probably do that test at some point, but if we think snaptree is acceptable I suggest we go with that at first since it is absolutely unclear the persistent sorted map will be faster (it will generate more garbage for instance) and I'm sure how long it will take to do such a test. Add row-level isolation --- Key: CASSANDRA-2893 URL: https://issues.apache.org/jira/browse/CASSANDRA-2893 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Sylvain Lebresne Priority: Minor Fix For: 1.1 Attachments: 0001-Move-deletion-infos-into-ISortedColumns-v2.patch, 0001-Move-deletion-infos-into-ISortedColumns.patch, 0002-Make-memtable-use-CF.addAll-v2.patch, 0002-Make-memtable-use-CF.addAll.patch, 0003-Add-AtomicSortedColumn-and-snapTree-v2.patch, 0003-Add-AtomicSortedColumn-and-snapTree.patch, latency-plain.svg, latency.svg, snaptree-0.1-SNAPSHOT.jar This could be done using an the atomic ConcurrentMap operations from the Memtable and something like http://code.google.com/p/pcollections/ to replace the ConcurrentSkipListMap in ThreadSafeSortedColumns. The trick is that pcollections does not provide a SortedMap, so we probably need to write our own. Googling [persistent sortedmap] I found http://code.google.com/p/actord/source/browse/trunk/actord/src/main/scala/ff/collection (in scala) and http://clojure.org/data_structures#Data Structures-Maps. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment
[ https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174958#comment-13174958 ] Sylvain Lebresne commented on CASSANDRA-3623: - When I was originally checking the compression code, I did a few quick stress tests under yourkit, and in cases where the data was small and thus entirely in page cache, the cost of creating a RandomAccessFile on each read was taking the majority of the time. Note that I'm not saying this to justify this ticket necessarily, since: # we could use an object pool to avoid that cost # my tests were really toy test, so they would need confirmation but mmapping does avoid this cost. Just saying. I'll also note that Snappy has a way to decompress data from direct ByteBuffer directly ([snappydoc|http://wiki.snappy-java.googlecode.com/hg/apidocs/org/xerial/snappy/Snappy.html#compress(java.nio.ByteBuffer, java.nio.ByteBuffer)]) so this could potentially avoid 1 copy (we would go from page cache to decompressed buffer directly). Of course we should look how well that works, but again, just to feed the discussion. use MMapedBuffer in CompressedSegmentedFile.getSegment -- Key: CASSANDRA-3623 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Labels: compression Fix For: 1.1 Attachments: 0001-MMaped-Compression-segmented-file.patch CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to use the MMap and hence a higher CPU on the nodes and higher latencies on reads. This ticket is to implement the TODO mentioned in CompressedRandomAccessReader // TODO refactor this to separate concept of buffer to avoid lots of read() syscalls and compression buffer but i think a separate class for the Buffer will be better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3663) Change syntax of cqlsh for creating column families to be more descriptive for comparator and default_validation
Change syntax of cqlsh for creating column families to be more descriptive for comparator and default_validation Key: CASSANDRA-3663 URL: https://issues.apache.org/jira/browse/CASSANDRA-3663 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Donald Smith Priority: Minor According to http://crlog.info/2011/09/17/cassandra-query-language-cql-v2-0-reference/#Column+Family+Options+%28optional%29 the syntax for creating column families in cqlsh uses keywords comparator and default_validation. Better, more descriptive names for these would be column_name_comparator and column_value_validation or perhaps better yet column_key_comparator and column_value_validation. Two other people on the cassandra users' mailing list agreed with this suggestion. The existing syntax is unclear and confusing to beginners. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment
[ https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174963#comment-13174963 ] Pavel Yaskevich commented on CASSANDRA-3623: I'd like to see test and numbers like we did for BRAF related tickets CASSANDRA-1902 and CASSANDRA-1714 to be convinced. use MMapedBuffer in CompressedSegmentedFile.getSegment -- Key: CASSANDRA-3623 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Labels: compression Fix For: 1.1 Attachments: 0001-MMaped-Compression-segmented-file.patch CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to use the MMap and hence a higher CPU on the nodes and higher latencies on reads. This ticket is to implement the TODO mentioned in CompressedRandomAccessReader // TODO refactor this to separate concept of buffer to avoid lots of read() syscalls and compression buffer but i think a separate class for the Buffer will be better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3663) Change syntax of cqlsh for creating column families to be more descriptive for comparator and default_validation
[ https://issues.apache.org/jira/browse/CASSANDRA-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174967#comment-13174967 ] Jonathan Ellis commented on CASSANDRA-3663: --- FYI, we use name with columns to avoid confusion with keys in rows. Change syntax of cqlsh for creating column families to be more descriptive for comparator and default_validation Key: CASSANDRA-3663 URL: https://issues.apache.org/jira/browse/CASSANDRA-3663 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Donald Smith Priority: Minor According to http://crlog.info/2011/09/17/cassandra-query-language-cql-v2-0-reference/#Column+Family+Options+%28optional%29 the syntax for creating column families in cqlsh uses keywords comparator and default_validation. Better, more descriptive names for these would be column_name_comparator and column_value_validation or perhaps better yet column_key_comparator and column_value_validation. Two other people on the cassandra users' mailing list agreed with this suggestion. The existing syntax is unclear and confusing to beginners. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3610) Checksum improvement for CompressedRandomAccessReader
[ https://issues.apache.org/jira/browse/CASSANDRA-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174968#comment-13174968 ] Jonathan Ellis commented on CASSANDRA-3610: --- Hmm, I'm not quite sure what to make of that table. Is it rendering correctly? Checksum improvement for CompressedRandomAccessReader - Key: CASSANDRA-3610 URL: https://issues.apache.org/jira/browse/CASSANDRA-3610 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Environment: JVM Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 1.1 Attachments: 0001-use-pure-java-CRC32-v2.patch, 0001-use-pure-java-CRC32.patch When compression is on, Currently we see checksum taking up about 40% of the CPU more than snappy library. Looks like hadoop solved it by implementing their own checksum, we can either use it or implement something like that. http://images.slidesharecdn.com/1toddlipconyanpeichen-cloudera-hadoopandperformance-final-10132228-phpapp01-slide-15-768.jpg?1321043717 in our test env it provided 50% improvement over native implementation which uses jni to call the OS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2805) Clean up mbeans that return Internal Cassandra types
[ https://issues.apache.org/jira/browse/CASSANDRA-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Bailey reassigned CASSANDRA-2805: -- Assignee: Nick Bailey (was: Gaurav Sharma) Clean up mbeans that return Internal Cassandra types Key: CASSANDRA-2805 URL: https://issues.apache.org/jira/browse/CASSANDRA-2805 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.1 Reporter: Nick Bailey Assignee: Nick Bailey Priority: Minor Labels: lhf Fix For: 1.1 We need to clean up wherever we return internal cassandra objects over jmx. Namely CompactionInfo objects as well as Tokens. There may be a few other examples. This is bad for two reasons 1. You have to load the cassandra jar when querying these mbeans, which sucks. 2. Stuff breaks between versions when things are moved. For example, CASSANDRA-1610 moves the compaction related classes around. Any code querying those jmx mbeans in 0.8.0 is now broken in 0.8.2. (assuming those moves stay in the 0.8 branch) For things like CompactionInfo we should just expose more mbean methods or serialize to something standard like json. I'd like to target this for 0.8.2. Since we've already broken compatibility between 0.8.0 and 0.8.1, I'd say just fix this everywhere now. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3663) Change syntax of cqlsh for creating column families to be more descriptive for comparator and default_validation
[ https://issues.apache.org/jira/browse/CASSANDRA-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174991#comment-13174991 ] Brandon Williams commented on CASSANDRA-3663: - bq. comparator and default_validation 'validation' is a noun, so 'validator' still makes more sense to me. Change syntax of cqlsh for creating column families to be more descriptive for comparator and default_validation Key: CASSANDRA-3663 URL: https://issues.apache.org/jira/browse/CASSANDRA-3663 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Donald Smith Priority: Minor According to http://crlog.info/2011/09/17/cassandra-query-language-cql-v2-0-reference/#Column+Family+Options+%28optional%29 the syntax for creating column families in cqlsh uses keywords comparator and default_validation. Better, more descriptive names for these would be column_name_comparator and column_value_validation or perhaps better yet column_key_comparator and column_value_validation. Two other people on the cassandra users' mailing list agreed with this suggestion. The existing syntax is unclear and confusing to beginners. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment
[ https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174992#comment-13174992 ] Vijay commented on CASSANDRA-3623: -- Pravel, I already have the hot methods (this patch removes the method org.apache.cassandra.io.util.RandomAccessReader.init(java.io.File, int, boolean)) before and after... i will do an updated patch without mem copy and share the same the results. use MMapedBuffer in CompressedSegmentedFile.getSegment -- Key: CASSANDRA-3623 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Reporter: Vijay Assignee: Vijay Labels: compression Fix For: 1.1 Attachments: 0001-MMaped-Compression-segmented-file.patch CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to use the MMap and hence a higher CPU on the nodes and higher latencies on reads. This ticket is to implement the TODO mentioned in CompressedRandomAccessReader // TODO refactor this to separate concept of buffer to avoid lots of read() syscalls and compression buffer but i think a separate class for the Buffer will be better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3655) NPE when running upgradesstables
[ https://issues.apache.org/jira/browse/CASSANDRA-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3655: -- Attachment: 3655.txt The stacktrace indicates that either the controller or the cf parameters have to be null. (key being null would NOT cause this exception.) Controller is always initialized to non-null except in one place in the Streaming code which should be irrelevant here. And cf comes from merge(rows) which should also return non-null unless there's a deserialization exception. I bet that's what's happening -- there should be errors in the log about Skipping row X in that case. Still, we shouldn't be ignoring those exceptions -- we should kill the compaction (or upgrade) and let the operator decide how to deal with them (e.g. with scrub). Patch to fix that behavior, clean up the Streaming controller use, and add extra assertions. NPE when running upgradesstables Key: CASSANDRA-3655 URL: https://issues.apache.org/jira/browse/CASSANDRA-3655 Project: Cassandra Issue Type: Bug Affects Versions: 1.0.5 Environment: 1.0.5 + patch for https://issues.apache.org/jira/browse/CASSANDRA-3618 Reporter: Tupshin Harper Assignee: Tupshin Harper Fix For: 1.0.7 Attachments: 3655.txt Running a test upgrade from 0.7(version f sstables) to 1.0. upgradesstables runs for about 40 minutes and then NPE's when trying to retrieve a key. No files have been succesfully upgraded. Likely related is that scrub (without having run upgrade) consumes all RAM and OOMs. Possible theory is that a lot of paths call IPartitioner's decorateKey, and, at least in the randompartitioner's implementation, if any of those callers pass a null ByteBuffer, they key will be null in the stack trace below. java.util.concurrent.ExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) at java.util.concurrent.FutureTask.get(FutureTask.java:83) at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203) at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219) at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:970) at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1540) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427) at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) at sun.rmi.transport.Transport$1.run(Transport.java:159) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:155) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) at
[jira] [Updated] (CASSANDRA-3655) NPE when running upgradesstables
[ https://issues.apache.org/jira/browse/CASSANDRA-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3655: -- Reviewer: slebresne Labels: compaction (was: ) NPE when running upgradesstables Key: CASSANDRA-3655 URL: https://issues.apache.org/jira/browse/CASSANDRA-3655 Project: Cassandra Issue Type: Bug Affects Versions: 1.0.5 Environment: 1.0.5 + patch for https://issues.apache.org/jira/browse/CASSANDRA-3618 Reporter: Tupshin Harper Assignee: Tupshin Harper Labels: compaction Fix For: 1.0.7 Attachments: 3655.txt Running a test upgrade from 0.7(version f sstables) to 1.0. upgradesstables runs for about 40 minutes and then NPE's when trying to retrieve a key. No files have been succesfully upgraded. Likely related is that scrub (without having run upgrade) consumes all RAM and OOMs. Possible theory is that a lot of paths call IPartitioner's decorateKey, and, at least in the randompartitioner's implementation, if any of those callers pass a null ByteBuffer, they key will be null in the stack trace below. java.util.concurrent.ExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) at java.util.concurrent.FutureTask.get(FutureTask.java:83) at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203) at org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:219) at org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:970) at org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:1540) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427) at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) at sun.rmi.transport.Transport$1.run(Transport.java:159) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:155) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.compaction.PrecompactedRow.removeDeletedAndOldShards(PrecompactedRow.java:65) at org.apache.cassandra.db.compaction.PrecompactedRow.init(PrecompactedRow.java:92) at org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:137) at org.apache.cassandra.db.compaction.CompactionIterable$Reducer.getReduced(CompactionIterable.java:102) at
svn commit: r1222408 - /cassandra/branches/cassandra-1.0/debian/control
Author: eevans
Date: Thu Dec 22 19:43:05 2011
New Revision: 1222408
URL: http://svn.apache.org/viewvc?rev=1222408view=rev
Log:
debian: revert to hardcode runtime dependencies
Patch by eevans for CASSANDRA-3458
Modified:
cassandra/branches/cassandra-1.0/debian/control
Modified: cassandra/branches/cassandra-1.0/debian/control
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/debian/control?rev=1222408r1=1222407r2=1222408view=diff
==
--- cassandra/branches/cassandra-1.0/debian/control (original)
+++ cassandra/branches/cassandra-1.0/debian/control Thu Dec 22 19:43:05 2011
@@ -11,7 +11,7 @@ Standards-Version: 3.8.3
Package: cassandra
Architecture: all
-Depends: openjdk-6-jre-headless (= 6b11) | java6-runtime, jsvc (= 1.0),
libcommons-daemon-java (= 1.0), adduser, libjna-java, python-cql,
${python:Depends}, ${misc:Depends}
+Depends: openjdk-6-jre-headless (= 6b11) | java6-runtime, jsvc (= 1.0),
libcommons-daemon-java (= 1.0), adduser, libjna-java, python-cql, python (=
2.5), python-support (= 0.90.0), ${misc:Depends}
Conflicts: apache-cassandra1
Replaces: apache-cassandra1
Description: distributed storage system for structured data
[jira] [Commented] (CASSANDRA-3661) Re-introduce timeout debug messages in CassandraServer
[
https://issues.apache.org/jira/browse/CASSANDRA-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175004#comment-13175004
]
Hudson commented on CASSANDRA-3661:
---
Integrated in Cassandra-0.8 #421 (See
[https://builds.apache.org/job/Cassandra-0.8/421/])
Re-introduce timeout debug messages in CassandraServer.
Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-3661
brandonwilliams :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1222372
Files :
*
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
Re-introduce timeout debug messages in CassandraServer
--
Key: CASSANDRA-3661
URL: https://issues.apache.org/jira/browse/CASSANDRA-3661
Project: Cassandra
Issue Type: Improvement
Components: API
Affects Versions: 0.8.0
Reporter: Jonathan Ellis
Assignee: Brandon Williams
Priority: Trivial
Labels: thrift
Fix For: 0.8.10, 1.0.7
Attachments: 3661.txt
In 0.7 we log at debug when returning TOE back to the client:
{code}
. catch (TimeoutException e)
{
logger.debug(... timed out);
throw new TimedOutException();
}
{code}
This is primarily useful when reading through debug logs to make it obvious
when StorageProxy gave up waiting and cleared its callbacks.
At some point this got removed from 0.8+. (Or possibly never got correctly
merged upwards.) Let's fix this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3610) Checksum improvement for CompressedRandomAccessReader
[ https://issues.apache.org/jira/browse/CASSANDRA-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175014#comment-13175014 ] Vijay commented on CASSANDRA-3610: -- Fields: 1 - bytes sent to checksum 2 - Mb/s (trials * size / 1024.0 / 1024.0) by pure java implementation 3 - Mb/s by Native CRC 5 - Pure Jave Random bytes from and till 6 - Pure Java Mb/S 8 - Native Random bytes from and till 9 - Native Java Mb/S Hope that makes sense... i should have separated the two tests random and constant byte test. Checksum improvement for CompressedRandomAccessReader - Key: CASSANDRA-3610 URL: https://issues.apache.org/jira/browse/CASSANDRA-3610 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Environment: JVM Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 1.1 Attachments: 0001-use-pure-java-CRC32-v2.patch, 0001-use-pure-java-CRC32.patch When compression is on, Currently we see checksum taking up about 40% of the CPU more than snappy library. Looks like hadoop solved it by implementing their own checksum, we can either use it or implement something like that. http://images.slidesharecdn.com/1toddlipconyanpeichen-cloudera-hadoopandperformance-final-10132228-phpapp01-slide-15-768.jpg?1321043717 in our test env it provided 50% improvement over native implementation which uses jni to call the OS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3610) Checksum improvement for CompressedRandomAccessReader
[ https://issues.apache.org/jira/browse/CASSANDRA-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175017#comment-13175017 ] Jonathan Ellis commented on CASSANDRA-3610: --- So, it looks like you're seeing similar results, that for larger values native is faster. Specifically, for arrays = 128 bytes. Checksum improvement for CompressedRandomAccessReader - Key: CASSANDRA-3610 URL: https://issues.apache.org/jira/browse/CASSANDRA-3610 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Environment: JVM Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 1.1 Attachments: 0001-use-pure-java-CRC32-v2.patch, 0001-use-pure-java-CRC32.patch When compression is on, Currently we see checksum taking up about 40% of the CPU more than snappy library. Looks like hadoop solved it by implementing their own checksum, we can either use it or implement something like that. http://images.slidesharecdn.com/1toddlipconyanpeichen-cloudera-hadoopandperformance-final-10132228-phpapp01-slide-15-768.jpg?1321043717 in our test env it provided 50% improvement over native implementation which uses jni to call the OS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3507) Proposal: separate cqlsh from CQL drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-3507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175025#comment-13175025 ] paul cannon commented on CASSANDRA-3507: Another solution proposed by eevans on irc: bq. 14:01 urandom my personal opinion would be that cqlsh should be a different project (whether that means coupled with the driver, or stand-alone). if that means that -cli is implicitly made more official when we'd rather it not be, then it should be removed bq. 14:01 urandom but then, I know I'm in the minority in thinking that we don't need to be quite so batteries included on everything Proposal: separate cqlsh from CQL drivers - Key: CASSANDRA-3507 URL: https://issues.apache.org/jira/browse/CASSANDRA-3507 Project: Cassandra Issue Type: Improvement Components: Packaging, Tools Affects Versions: 1.0.3 Environment: Debian-based systems Reporter: paul cannon Assignee: paul cannon Priority: Minor Labels: cql, cqlsh Fix For: 1.0.7 Whereas: * It has been shown to be very desirable to decouple the release cycles of Cassandra from the various client CQL drivers, and * It is also desirable to include a good interactive CQL client with releases of Cassandra, and * It is not desirable for Cassandra releases to depend on 3rd-party software which is neither bundled with Cassandra nor readily available for every target platform, but * Any good interactive CQL client will require a CQL driver; Therefore, be it resolved that: * cqlsh will not use an official or supported CQL driver, but will include its own private CQL driver, not intended for use by anything else, and * the Cassandra project will still recommend installing and using a proper CQL driver for client software. To ease maintenance, the private CQL driver included with cqlsh may very well be created by copying the python CQL driver from one directory into another, but the user shouldn't rely on this. Maybe we even ought to take some minor steps to discourage its use for other purposes. Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3656) GC can take 0 ms
[ https://issues.apache.org/jira/browse/CASSANDRA-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3656: -- Attachment: 3656.txt GC can take 0 ms Key: CASSANDRA-3656 URL: https://issues.apache.org/jira/browse/CASSANDRA-3656 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.9, 0.8.7 Reporter: Jonathan Ellis Priority: Trivial Fix For: 0.8.10, 1.0.7 Attachments: 3656.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1222419 - in /cassandra/branches/cassandra-1.0: ./ src/java/org/apache/cassandra/service/ src/java/org/apache/cassandra/tools/
Author: jbellis
Date: Thu Dec 22 20:15:40 2011
New Revision: 1222419
URL: http://svn.apache.org/viewvc?rev=1222419view=rev
Log:
add nodetool setstreamthroughput
patch by Peter Schuller; reviewed by yukim for CASSANDRA-3571
Modified:
cassandra/branches/cassandra-1.0/CHANGES.txt
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageService.java
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageServiceMBean.java
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/tools/NodeCmd.java
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/tools/NodeProbe.java
Modified: cassandra/branches/cassandra-1.0/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/CHANGES.txt?rev=1222419r1=1222418r2=1222419view=diff
==
--- cassandra/branches/cassandra-1.0/CHANGES.txt (original)
+++ cassandra/branches/cassandra-1.0/CHANGES.txt Thu Dec 22 20:15:40 2011
@@ -1,4 +1,5 @@
1.0.7
+ * add nodetool setstreamthroughput (CASSANDRA-3571)
* fix assertion when dropping a columnfamily with no sstables (CASSANDRA-3614)
* more efficient allocation of small bloom filters (CASSANDRA-3618)
* CLibrary.createHardLinkWithExec() to check for errors (CASSANDRA-3101)
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageService.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageService.java?rev=1222419r1=1222418r2=1222419view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageService.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageService.java
Thu Dec 22 20:15:40 2011
@@ -634,6 +634,17 @@ public class StorageService implements I
return joined;
}
+public void setStreamThroughputMbPerSec(int value)
+{
+DatabaseDescriptor.setStreamThroughputOutboundMegabitsPerSec(value);
+logger_.info(setstreamthroughput: throttle set to {}, value);
+}
+
+public int getStreamThroughputMbPerSec()
+{
+return DatabaseDescriptor.getStreamThroughputOutboundMegabitsPerSec();
+}
+
public int getCompactionThroughputMbPerSec()
{
return DatabaseDescriptor.getCompactionThroughputMbPerSec();
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageServiceMBean.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageServiceMBean.java?rev=1222419r1=1222418r2=1222419view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageServiceMBean.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageServiceMBean.java
Thu Dec 22 20:15:40 2011
@@ -351,6 +351,9 @@ public interface StorageServiceMBean
public boolean isJoined();
public int getExceptionCount();
+
+public void setStreamThroughputMbPerSec(int value);
+public int getStreamThroughputMbPerSec();
public int getCompactionThroughputMbPerSec();
public void setCompactionThroughputMbPerSec(int value);
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/tools/NodeCmd.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/tools/NodeCmd.java?rev=1222419r1=1222418r2=1222419view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/tools/NodeCmd.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/tools/NodeCmd.java
Thu Dec 22 20:15:40 2011
@@ -108,6 +108,7 @@ public class NodeCmd
SETCACHECAPACITY,
SETCOMPACTIONTHRESHOLD,
SETCOMPACTIONTHROUGHPUT,
+SETSTREAMTHROUGHPUT,
SNAPSHOT,
STATUSTHRIFT,
TPSTATS,
@@ -147,6 +148,7 @@ public class NodeCmd
addCmdHelp(header, move new token, Move node on the token ring to
a new token);
addCmdHelp(header, removetoken status|force|token, Show status of
current token removal, force completion of pending removal or remove providen
token);
addCmdHelp(header, setcompactionthroughput value_in_mb, Set the
MB/s throughput cap for compaction in the system, or 0 to disable throttling.);
+addCmdHelp(header, setstreamthroughput value_in_mb, Set the MB/s
throughput cap for streaming in the system, or 0 to disable throttling.);
addCmdHelp(header, describering [keyspace], Shows the token ranges
info of a given keyspace.);
// Two args
@@ -694,6 +696,11 @@ public class
[jira] [Updated] (CASSANDRA-3571) make stream throttling configurable at runtime with nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3571: -- Component/s: Tools Labels: jmx (was: ) make stream throttling configurable at runtime with nodetool Key: CASSANDRA-3571 URL: https://issues.apache.org/jira/browse/CASSANDRA-3571 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Peter Schuller Assignee: Peter Schuller Priority: Minor Labels: jmx Fix For: 1.0.7 Attachments: CASSANDRA-3571-1.0-rebased-v2.txt, CASSANDRA-3571-1.0-rebased.txt, CASSANDRA-3571-1.0.txt Attaching patch that does this, against 1.0. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3452) Create an 'infinite bootstrap' mode for sampling live traffic
[ https://issues.apache.org/jira/browse/CASSANDRA-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175029#comment-13175029 ] Jonathan Ellis commented on CASSANDRA-3452: --- +1 (I'd be okay with 01 in 1.0.7, but 02 should be 1.1 IMO) Create an 'infinite bootstrap' mode for sampling live traffic - Key: CASSANDRA-3452 URL: https://issues.apache.org/jira/browse/CASSANDRA-3452 Project: Cassandra Issue Type: New Feature Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1 Attachments: 0001-Ability-to-set-compaction-strategy-via-mbean.txt, 0002-Allow-survey-only-mode-after-bootstrapping.txt You may want to, for example, test a new compaction strategy with live traffic to see how it will fare. In this mode, the node would follow the bootstrap procedure as normal, but never fully join the ring. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3656) GC can take 0 ms
[ https://issues.apache.org/jira/browse/CASSANDRA-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175030#comment-13175030 ] Brandon Williams commented on CASSANDRA-3656: - +1 GC can take 0 ms Key: CASSANDRA-3656 URL: https://issues.apache.org/jira/browse/CASSANDRA-3656 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.9, 0.8.7 Reporter: Jonathan Ellis Priority: Trivial Fix For: 0.8.10, 1.0.7 Attachments: 3656.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1222421 - in /cassandra/trunk: ./ contrib/ debian/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/service/ src/java
Author: jbellis
Date: Thu Dec 22 20:21:29 2011
New Revision: 1222421
URL: http://svn.apache.org/viewvc?rev=1222421view=rev
Log:
merge from 1.0
Modified:
cassandra/trunk/ (props changed)
cassandra/trunk/CHANGES.txt
cassandra/trunk/contrib/ (props changed)
cassandra/trunk/debian/control
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
(props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
(props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
(props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
(props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
(props changed)
cassandra/trunk/src/java/org/apache/cassandra/db/SystemTable.java
cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
cassandra/trunk/src/java/org/apache/cassandra/service/StorageServiceMBean.java
cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java
cassandra/trunk/src/java/org/apache/cassandra/tools/NodeCmd.java
cassandra/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java
Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Thu Dec 22 20:21:29 2011
@@ -1,10 +1,10 @@
/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291
/cassandra/branches/cassandra-0.7:1026516-1211709
/cassandra/branches/cassandra-0.7.0:1053690-1055654
-/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916
+/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916,1222372
/cassandra/branches/cassandra-0.8.0:1125021-1130369
/cassandra/branches/cassandra-0.8.1:1101014-1125018
-/cassandra/branches/cassandra-1.0:1167085-1222319
+/cassandra/branches/cassandra-1.0:1167085-1222420
/cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243
/cassandra/branches/cassandra-1.0.5:1208016
/cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689
Modified: cassandra/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1222421r1=1222420r2=1222421view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Thu Dec 22 20:21:29 2011
@@ -30,6 +30,7 @@
1.0.7
+ * add nodetool setstreamthroughput (CASSANDRA-3571)
* fix assertion when dropping a columnfamily with no sstables (CASSANDRA-3614)
* more efficient allocation of small bloom filters (CASSANDRA-3618)
* CLibrary.createHardLinkWithExec() to check for errors (CASSANDRA-3101)
Propchange: cassandra/trunk/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Thu Dec 22 20:21:29 2011
@@ -1,10 +1,10 @@
/cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009
/cassandra/branches/cassandra-0.7/contrib:1026516-1211709
/cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654
-/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916
+/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916,1222372
/cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369
/cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018
-/cassandra/branches/cassandra-1.0/contrib:1167085-1222319
+/cassandra/branches/cassandra-1.0/contrib:1167085-1222420
/cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243
/cassandra/branches/cassandra-1.0.5/contrib:1208016
/cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689
Modified: cassandra/trunk/debian/control
URL:
http://svn.apache.org/viewvc/cassandra/trunk/debian/control?rev=1222421r1=1222420r2=1222421view=diff
==
--- cassandra/trunk/debian/control (original)
+++ cassandra/trunk/debian/control Thu Dec 22 20:21:29 2011
@@ -11,7 +11,7 @@ Standards-Version: 3.8.3
Package: cassandra
Architecture: all
-Depends: openjdk-6-jre-headless (= 6b11) | java6-runtime, jsvc (= 1.0),
libcommons-daemon-java (= 1.0), adduser, libjna-java, python-cql,
${python:Depends}, ${misc:Depends}
+Depends: openjdk-6-jre-headless (= 6b11) | java6-runtime, jsvc (= 1.0),
libcommons-daemon-java (= 1.0), adduser, libjna-java, python-cql, python (=
2.5), python-support (= 0.90.0), ${misc:Depends}
Conflicts: apache-cassandra1
Replaces:
svn commit: r1222423 - in /cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db: ColumnFamilyStore.java ColumnFamilyStoreMBean.java
Author: brandonwilliams
Date: Thu Dec 22 20:23:48 2011
New Revision: 1222423
URL: http://svn.apache.org/viewvc?rev=1222423view=rev
Log:
Allow setting compaction strategy via JMX.
Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-3452
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1222423r1=1222422r2=1222423view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Thu Dec 22 20:23:48 2011
@@ -204,6 +204,17 @@ public class ColumnFamilyStore implement
}
}
+public void setCompactionStrategyClass(String compactionStrategyClass)
throws ConfigurationException
+{
+metadata.compactionStrategyClass =
CFMetaData.createCompactionStrategy(compactionStrategyClass);
+maybeReloadCompactionStrategy();
+}
+
+public String getCompactionStrategyClass()
+{
+return metadata.compactionStrategyClass.getName();
+}
+
private ColumnFamilyStore(Table table, String columnFamilyName,
IPartitioner partitioner, int generation, CFMetaData metadata)
{
assert metadata != null : null metadata for + table + : +
columnFamilyName;
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java?rev=1222423r1=1222422r2=1222423view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
Thu Dec 22 20:23:48 2011
@@ -207,6 +207,17 @@ public interface ColumnFamilyStoreMBean
public void setMaximumCompactionThreshold(int threshold);
/**
+ * Sets the compaction strategy by class name
+ * @param className the name of the compaction strategy class
+ */
+public void setCompactionStrategyClass(String className) throws
ConfigurationException;
+
+/**
+ * Gets the compaction strategy class name
+ */
+public String getCompactionStrategyClass();
+
+/**
* Disable automatic compaction.
*/
public void disableAutoCompaction();
svn commit: r1222425 - /cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
Author: brandonwilliams
Date: Thu Dec 22 20:25:10 2011
New Revision: 1222425
URL: http://svn.apache.org/viewvc?rev=1222425view=rev
Log:
Add 'write survey' mode that bootstraps but does not join.
Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-3452
Modified:
cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
Modified:
cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java?rev=1222425r1=1222424r2=1222425view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
Thu Dec 22 20:25:10 2011
@@ -211,6 +211,10 @@ public class StorageService implements I
/* Are we starting this node in bootstrap mode? */
private boolean isBootstrapMode;
+
+/* we bootstrap but do NOT join the ring unless told to do so */
+private boolean isSurveyMode=
Boolean.parseBoolean(System.getProperty(cassandra.write_survey, false));
+
/* when intialized as a client, we shouldn't write to the system table. */
private boolean isClientMode;
private boolean initialized;
@@ -610,14 +614,21 @@ public class StorageService implements I
}
}
-// start participating in the ring.
-SystemTable.setBootstrapped(true);
-setToken(token);
-// remove the existing info about the replaced node.
-if (current != null)
-Gossiper.instance.replacedEndpoint(current);
-logger_.info(Bootstrap/Replace/Move completed! Now serving reads.);
-assert tokenMetadata_.sortedTokens().size() 0;
+if (!isSurveyMode)
+{
+// start participating in the ring.
+SystemTable.setBootstrapped(true);
+setToken(token);
+// remove the existing info about the replaced node.
+if (current != null)
+Gossiper.instance.replacedEndpoint(current);
+logger_.info(Bootstrap/Replace/Move completed! Now serving
reads.);
+assert tokenMetadata_.sortedTokens().size() 0;
+}
+else
+{
+logger_.info(Bootstrap complete, but write survey mode is active,
not becoming an active ring member. Use JMX (StorageService-joinRing()) to
finalize ring joining.);
+}
}
public synchronized void joinRing() throws IOException,
org.apache.cassandra.config.ConfigurationException
@@ -627,6 +638,14 @@ public class StorageService implements I
logger_.info(Joining ring by operator request);
joinTokenRing(0);
}
+else if (isSurveyMode)
+{
+setToken(SystemTable.getSavedToken());
+SystemTable.setBootstrapped(true);
+isSurveyMode = false;
+logger_.info(Leaving write survey mode and joining ring at
operator request);
+assert tokenMetadata_.sortedTokens().size() 0;
+}
}
public boolean isJoined()
[jira] [Commented] (CASSANDRA-3610) Checksum improvement for CompressedRandomAccessReader
[ https://issues.apache.org/jira/browse/CASSANDRA-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175035#comment-13175035 ] Vijay commented on CASSANDRA-3610: -- Ooops pasted the wrong data the above data is without any Heap settings hence GC becomes a bottleneck... Plz see the below :) /usr/java/latest/jre/bin/java -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms8G -Xmx8G -Xmn2G -XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -jar TestCRC32Performance.jar ||bytes||PureJava MB/sec||Native MB/sec||Random PureJava MB/sec||Native MB/sec|| | 1 |121.124|11.866 | | 2 |161.981|23.851 | | 4 |204.718|45.486 | | 8 |297.229|76.296 | | 16|379.268|117.326| | 32|440.153|157.711| | 64|468.143|193.304|| PureJava |0-64 |272.921 MB/sec|| Native|0-64 |145.289 MB/sec| | 128 |500.006|219.657|| PureJava |0-128 |367.816 MB/sec|| Native|0-128 |186.861 MB/sec| | 256 |511.572|234.052|| PureJava |0-256 |432.433 MB/sec|| Native|0-256 |214.047 MB/sec| | 512 |517.550|242.634|| PureJava |0-512 |474.074 MB/sec|| Native|0-512 |231.047 MB/sec| | 1024 |516.994|246.424|| PureJava |0-1024 |498.055 MB/sec|| Native|0-1024 |241.056 MB/sec| | 2048 |518.095|248.529|| PureJava |0-2048 |509.960 MB/sec|| Native|0-2048 |245.683 MB/sec| | 4096 |522.002|249.755|| PureJava |0-4096 |518.226 MB/sec|| Native|0-4096 |248.062 MB/sec| | 8192 |522.795|250.316|| PureJava |0-8192 |520.326 MB/sec|| Native|0-8192 |249.519 MB/sec| | 16384 |522.521|250.484|| PureJava |0-16384 |522.480 MB/sec|| Native|0-16384|250.002 MB/sec| | 32768 |521.098|250.604|| PureJava |0-32768 |520.349 MB/sec|| Native|0-32768|250.494 MB/sec| | 65536 |520.973|250.837|| PureJava |0-65536 |520.392 MB/sec|| Native|0-65536|249.063 MB/sec| | 131072|510.129|248.949|| PureJava |0-131072 |516.246 MB/sec|| Native|0-131072 |249.535 MB/sec| | 262144|513.534|249.506|| PureJava |0-262144 |514.407 MB/sec|| Native|0-262144 |250.617 MB/sec| | 524288|519.554|250.696|| PureJava |0-524288 |520.402 MB/sec|| Native|0-524288 |251.048 MB/sec| | 1048576 |519.559|250.557|| PureJava |0-1048576 |520.403 MB/sec|| Native|0-1048576 |250.734 MB/sec| | 2097152 |519.259|250.456|| PureJava |0-2097152 |519.337 MB/sec|| Native|0-2097152 |250.299 MB/sec| | 4194304 |518.649|250.470|| PureJava |0-4194304 |518.495 MB/sec|| Native|0-4194304 |250.523 MB/sec| | 8388608 |501.986|248.044|| PureJava |0-8388608 |509.521 MB/sec|| Native|0-8388608 |248.626 MB/sec| | 16777216 |508.201|247.587|| PureJava |0-16777216 |505.258 MB/sec|| Native|0-16777216 |249.558 MB/sec| Checksum improvement for CompressedRandomAccessReader - Key: CASSANDRA-3610 URL: https://issues.apache.org/jira/browse/CASSANDRA-3610 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Environment: JVM Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 1.1 Attachments: 0001-use-pure-java-CRC32-v2.patch, 0001-use-pure-java-CRC32.patch When compression is on, Currently we see checksum taking up about 40% of the CPU more than snappy library. Looks like hadoop solved it by implementing their own checksum, we can either use it or implement something like that. http://images.slidesharecdn.com/1toddlipconyanpeichen-cloudera-hadoopandperformance-final-10132228-phpapp01-slide-15-768.jpg?1321043717 in our test env it provided 50% improvement over native implementation which uses jni to call the OS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3610) Checksum improvement for CompressedRandomAccessReader
[ https://issues.apache.org/jira/browse/CASSANDRA-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175035#comment-13175035 ] Vijay edited comment on CASSANDRA-3610 at 12/22/11 8:29 PM: Ooops pasted the wrong data the above data is without any Heap settings hence GC becomes a bottleneck... Plz see the below :) /usr/java/latest/jre/bin/java -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms8G -Xmx8G -Xmn2G -XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -jar TestCRC32Performance.jar ||bytes||PureJava MB/sec||Native MB/sec||Random PureJava MB/sec||Native MB/sec|| | 1 |121.124|11.866 | | 2 |161.981|23.851 | | 4 |204.718|45.486 | | 8 |297.229|76.296 | | 16|379.268|117.326| | 32|440.153|157.711| | 64|468.143|193.304|| PureJava |0-64 |272.921 MB/sec|| Native|0-64 |145.289 MB/sec| | 128 |500.006|219.657|| PureJava |0-128 |367.816 MB/sec|| Native|0-128 |186.861 MB/sec| | 256 |511.572|234.052|| PureJava |0-256 |432.433 MB/sec|| Native|0-256 |214.047 MB/sec| | 512 |517.550|242.634|| PureJava |0-512 |474.074 MB/sec|| Native|0-512 |231.047 MB/sec| | 1024 |516.994|246.424|| PureJava |0-1024 |498.055 MB/sec|| Native|0-1024 |241.056 MB/sec| | 2048 |518.095|248.529|| PureJava |0-2048 |509.960 MB/sec|| Native|0-2048 |245.683 MB/sec| | 4096 |522.002|249.755|| PureJava |0-4096 |518.226 MB/sec|| Native|0-4096 |248.062 MB/sec| | 8192 |522.795|250.316|| PureJava |0-8192 |520.326 MB/sec|| Native|0-8192 |249.519 MB/sec| | 16384 |522.521|250.484|| PureJava |0-16384 |522.480 MB/sec|| Native|0-16384|250.002 MB/sec| | 32768 |521.098|250.604|| PureJava |0-32768 |520.349 MB/sec|| Native|0-32768|250.494 MB/sec| | 65536 |520.973|250.837|| PureJava |0-65536 |520.392 MB/sec|| Native|0-65536|249.063 MB/sec| | 131072|510.129|248.949|| PureJava |0-131072 |516.246 MB/sec|| Native|0-131072 |249.535 MB/sec| | 262144|513.534|249.506|| PureJava |0-262144 |514.407 MB/sec|| Native|0-262144 |250.617 MB/sec| | 524288|519.554|250.696|| PureJava |0-524288 |520.402 MB/sec|| Native|0-524288 |251.048 MB/sec| | 1048576 |519.559|250.557|| PureJava |0-1048576 |520.403 MB/sec|| Native|0-1048576 |250.734 MB/sec| | 2097152 |519.259|250.456|| PureJava |0-2097152 |519.337 MB/sec|| Native|0-2097152 |250.299 MB/sec| | 4194304 |518.649|250.470|| PureJava |0-4194304 |518.495 MB/sec|| Native|0-4194304 |250.523 MB/sec| | 8388608 |501.986|248.044|| PureJava |0-8388608 |509.521 MB/sec|| Native|0-8388608 |248.626 MB/sec| | 16777216 |508.201|247.587|| PureJava |0-16777216 |505.258 MB/sec|| Native|0-16777216 |249.558 MB/sec| [vijay_tcasstest@vijay_tcass--1a-i-aad629c8 ~]$ /usr/java/latest/jre/bin/java -version java version 1.6.0_27 Java(TM) SE Runtime Environment (build 1.6.0_27-b07) Java HotSpot(TM) 64-Bit Server VM (build 20.2-b06, mixed mode) [vijay_tcasstest@vijay_tcass--1a-i-aad629c8 ~]$ was (Author: vijay2...@yahoo.com): Ooops pasted the wrong data the above data is without any Heap settings hence GC becomes a bottleneck... Plz see the below :) /usr/java/latest/jre/bin/java -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms8G -Xmx8G -Xmn2G -XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -jar TestCRC32Performance.jar ||bytes||PureJava MB/sec||Native MB/sec||Random PureJava MB/sec||Native MB/sec|| | 1 |121.124|11.866 | | 2 |161.981|23.851 | | 4 |204.718|45.486 | | 8 |297.229|76.296 | | 16|379.268|117.326| | 32|440.153|157.711| | 64|468.143|193.304|| PureJava |0-64 |272.921 MB/sec|| Native|0-64 |145.289 MB/sec| | 128 |500.006|219.657|| PureJava |0-128
[jira] [Updated] (CASSANDRA-3554) Hints are not replayed unless node was marked down
[ https://issues.apache.org/jira/browse/CASSANDRA-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3554: -- Attachment: 3554-1.0-v2.txt You're right, we switched from InetAddress as the key to Tokens. v2 attached. Hints are not replayed unless node was marked down -- Key: CASSANDRA-3554 URL: https://issues.apache.org/jira/browse/CASSANDRA-3554 Project: Cassandra Issue Type: Bug Affects Versions: 1.0.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Labels: hintedhandoff, jmx Fix For: 1.0.7 Attachments: 0001-cleanup.patch, 0002-deliver.patch, 3554-1.0-v2.txt, 3554-1.0.txt If B drops a write from A because it is overwhelmed (but not dead), A will hint the write. But it will never get notified that B is back up (since it was never down), so it will never attempt hint delivery. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3610) Checksum improvement for CompressedRandomAccessReader
[ https://issues.apache.org/jira/browse/CASSANDRA-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175035#comment-13175035 ] Vijay edited comment on CASSANDRA-3610 at 12/22/11 8:30 PM: Ooops pasted the wrong data the above data is without any Heap settings and on Open JDK.. Plz see the below :) /usr/java/latest/jre/bin/java -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms8G -Xmx8G -Xmn2G -XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -jar TestCRC32Performance.jar ||bytes||PureJava MB/sec||Native MB/sec||Random PureJava MB/sec||Native MB/sec|| | 1 |121.124|11.866 | | 2 |161.981|23.851 | | 4 |204.718|45.486 | | 8 |297.229|76.296 | | 16|379.268|117.326| | 32|440.153|157.711| | 64|468.143|193.304|| PureJava |0-64 |272.921 MB/sec|| Native|0-64 |145.289 MB/sec| | 128 |500.006|219.657|| PureJava |0-128 |367.816 MB/sec|| Native|0-128 |186.861 MB/sec| | 256 |511.572|234.052|| PureJava |0-256 |432.433 MB/sec|| Native|0-256 |214.047 MB/sec| | 512 |517.550|242.634|| PureJava |0-512 |474.074 MB/sec|| Native|0-512 |231.047 MB/sec| | 1024 |516.994|246.424|| PureJava |0-1024 |498.055 MB/sec|| Native|0-1024 |241.056 MB/sec| | 2048 |518.095|248.529|| PureJava |0-2048 |509.960 MB/sec|| Native|0-2048 |245.683 MB/sec| | 4096 |522.002|249.755|| PureJava |0-4096 |518.226 MB/sec|| Native|0-4096 |248.062 MB/sec| | 8192 |522.795|250.316|| PureJava |0-8192 |520.326 MB/sec|| Native|0-8192 |249.519 MB/sec| | 16384 |522.521|250.484|| PureJava |0-16384 |522.480 MB/sec|| Native|0-16384|250.002 MB/sec| | 32768 |521.098|250.604|| PureJava |0-32768 |520.349 MB/sec|| Native|0-32768|250.494 MB/sec| | 65536 |520.973|250.837|| PureJava |0-65536 |520.392 MB/sec|| Native|0-65536|249.063 MB/sec| | 131072|510.129|248.949|| PureJava |0-131072 |516.246 MB/sec|| Native|0-131072 |249.535 MB/sec| | 262144|513.534|249.506|| PureJava |0-262144 |514.407 MB/sec|| Native|0-262144 |250.617 MB/sec| | 524288|519.554|250.696|| PureJava |0-524288 |520.402 MB/sec|| Native|0-524288 |251.048 MB/sec| | 1048576 |519.559|250.557|| PureJava |0-1048576 |520.403 MB/sec|| Native|0-1048576 |250.734 MB/sec| | 2097152 |519.259|250.456|| PureJava |0-2097152 |519.337 MB/sec|| Native|0-2097152 |250.299 MB/sec| | 4194304 |518.649|250.470|| PureJava |0-4194304 |518.495 MB/sec|| Native|0-4194304 |250.523 MB/sec| | 8388608 |501.986|248.044|| PureJava |0-8388608 |509.521 MB/sec|| Native|0-8388608 |248.626 MB/sec| | 16777216 |508.201|247.587|| PureJava |0-16777216 |505.258 MB/sec|| Native|0-16777216 |249.558 MB/sec| [vijay_tcasstest@vijay_tcass--1a-i-aad629c8 ~]$ /usr/java/latest/jre/bin/java -version java version 1.6.0_27 Java(TM) SE Runtime Environment (build 1.6.0_27-b07) Java HotSpot(TM) 64-Bit Server VM (build 20.2-b06, mixed mode) [vijay_tcasstest@vijay_tcass--1a-i-aad629c8 ~]$ was (Author: vijay2...@yahoo.com): Ooops pasted the wrong data the above data is without any Heap settings hence GC becomes a bottleneck... Plz see the below :) /usr/java/latest/jre/bin/java -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms8G -Xmx8G -Xmn2G -XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -jar TestCRC32Performance.jar ||bytes||PureJava MB/sec||Native MB/sec||Random PureJava MB/sec||Native MB/sec|| | 1 |121.124|11.866 | | 2 |161.981|23.851 | | 4 |204.718|45.486 | | 8 |297.229|76.296 | | 16|379.268|117.326| | 32|440.153|157.711| | 64|468.143|193.304|| PureJava |0-64 |272.921 MB/sec|| Native|0-64 |145.289 MB/sec| | 128 |500.006|219.657|| PureJava |0-128 |367.816 MB/sec||
svn commit: r1222427 - /cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
Author: jbellis
Date: Thu Dec 22 20:32:31 2011
New Revision: 1222427
URL: http://svn.apache.org/viewvc?rev=1222427view=rev
Log:
improve message logged for stream protocol mismatch
patch by Peter Schuller; reviewed by jbellis for CASSANDRA-3652
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java?rev=1222427r1=1222426r2=1222427view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
Thu Dec 22 20:32:31 2011
@@ -82,8 +82,9 @@ public class IncomingTcpConnection exten
}
else
{
-// streaming connections are per-session and have a fixed
version. we can't do anything with a new-version stream connection, so drop it.
-logger.error(Received untranslated stream from newer
protocol version. Terminating connection!);
+// streaming connections are per-session and have a fixed
version. we can't do anything with a wrong-version stream connection, so drop
it.
+logger.error(Received stream using protocol version {}
(my version {}). Terminating connection,
+ version, MessagingService.version_);
}
// We are done with this connection
return;
[jira] [Commented] (CASSANDRA-3507) Proposal: separate cqlsh from CQL drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-3507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175040#comment-13175040 ] Jonathan Ellis commented on CASSANDRA-3507: --- I'd be okay with spinning both cli and cqlsh out to separate projects. Proposal: separate cqlsh from CQL drivers - Key: CASSANDRA-3507 URL: https://issues.apache.org/jira/browse/CASSANDRA-3507 Project: Cassandra Issue Type: Improvement Components: Packaging, Tools Affects Versions: 1.0.3 Environment: Debian-based systems Reporter: paul cannon Assignee: paul cannon Priority: Minor Labels: cql, cqlsh Fix For: 1.0.7 Whereas: * It has been shown to be very desirable to decouple the release cycles of Cassandra from the various client CQL drivers, and * It is also desirable to include a good interactive CQL client with releases of Cassandra, and * It is not desirable for Cassandra releases to depend on 3rd-party software which is neither bundled with Cassandra nor readily available for every target platform, but * Any good interactive CQL client will require a CQL driver; Therefore, be it resolved that: * cqlsh will not use an official or supported CQL driver, but will include its own private CQL driver, not intended for use by anything else, and * the Cassandra project will still recommend installing and using a proper CQL driver for client software. To ease maintenance, the private CQL driver included with cqlsh may very well be created by copying the python CQL driver from one directory into another, but the user shouldn't rely on this. Maybe we even ought to take some minor steps to discourage its use for other purposes. Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1222436 - /cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
Author: jbellis
Date: Thu Dec 22 20:39:58 2011
New Revision: 1222436
URL: http://svn.apache.org/viewvc?rev=1222436view=rev
Log:
throw ConfigurationException if chunk_length_kb would overflow
patch by pschuller; reviewed by jbellis for CASSANDRA-3644
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/compress/CompressionParameters.java?rev=1222436r1=1222435r2=1222436view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
Thu Dec 22 20:39:58 2011
@@ -150,7 +150,10 @@ public class CompressionParameters
try
{
-return 1024 * Integer.parseInt(chLengthKB);
+int parsed = Integer.parseInt(chLengthKB);
+if (parsed Integer.MAX_VALUE / 1024)
+throw new ConfigurationException(Value of + CHUNK_LENGTH_KB
+ is too large ( + parsed + ));
+return 1024 * parsed;
}
catch (NumberFormatException e)
{
svn commit: r1222437 - in /cassandra/branches/cassandra-1.0: CHANGES.txt src/java/org/apache/cassandra/service/GCInspector.java
Author: jbellis Date: Thu Dec 22 20:41:30 2011 New Revision: 1222437 URL: http://svn.apache.org/viewvc?rev=1222437view=rev Log: avoid logging (harmless) exception when GC takes 1ms patch by jbellis; reviewed by brandonwilliams for CASSANDRA-3656 Modified: cassandra/branches/cassandra-1.0/CHANGES.txt cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/GCInspector.java Modified: cassandra/branches/cassandra-1.0/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/CHANGES.txt?rev=1222437r1=1222436r2=1222437view=diff == --- cassandra/branches/cassandra-1.0/CHANGES.txt (original) +++ cassandra/branches/cassandra-1.0/CHANGES.txt Thu Dec 22 20:41:30 2011 @@ -1,4 +1,5 @@ 1.0.7 + * avoid logging (harmless) exception when GC takes 1ms (CASSANDRA-3656) * add nodetool setstreamthroughput (CASSANDRA-3571) * fix assertion when dropping a columnfamily with no sstables (CASSANDRA-3614) * more efficient allocation of small bloom filters (CASSANDRA-3618) Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/GCInspector.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/GCInspector.java?rev=1222437r1=1222436r2=1222437view=diff == --- cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/GCInspector.java (original) +++ cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/GCInspector.java Thu Dec 22 20:41:30 2011 @@ -99,8 +99,7 @@ public class GCInspector if (previousTotal.equals(total)) continue; gctimes.put(gc.getName(), total); -Long duration = total - previousTotal; -assert duration 0; +Long duration = total - previousTotal; // may be zero for a really fast collection Long previousCount = gccounts.get(gc.getName()); Long count = gc.getCollectionCount();
svn commit: r1222440 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/service/GCInspector.java
Author: jbellis Date: Thu Dec 22 20:42:23 2011 New Revision: 1222440 URL: http://svn.apache.org/viewvc?rev=1222440view=rev Log: avoid logging (harmless) exception when GC takes 1ms patch by jbellis; reviewed by brandonwilliams for CASSANDRA-3656 Modified: cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/GCInspector.java Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1222440r1=1222439r2=1222440view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Thu Dec 22 20:42:23 2011 @@ -2,6 +2,7 @@ * prevent new nodes from thinking down nodes are up forever (CASSANDRA-3626) 0.8.9 + * avoid logging (harmless) exception when GC takes 1ms (CASSANDRA-3656) * remove invalid assertion that table was opened before dropping it (CASSANDRA-3580) * range and index scans now only send requests to enough replicas to Modified: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/GCInspector.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/GCInspector.java?rev=1222440r1=1222439r2=1222440view=diff == --- cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/GCInspector.java (original) +++ cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/GCInspector.java Thu Dec 22 20:42:23 2011 @@ -98,8 +98,7 @@ public class GCInspector if (previousTotal.equals(total)) continue; gctimes.put(gc.getName(), total); -Long duration = total - previousTotal; -assert duration 0; +Long duration = total - previousTotal; // may be zero for a really fast collection Long previousCount = gccounts.get(gc.getName()); Long count = gc.getCollectionCount();
