[jira] [Updated] (CASSANDRA-4155) Make possible to authenticate user when loading data to Cassandra with BulkRecordWriter.
[ https://issues.apache.org/jira/browse/CASSANDRA-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4155: Fix Version/s: 1.1.1 Thanks for the patch, Michal! I'm fine with the 'nicer' version, since the only place ExternalClient is instantiated is in BOF and multiple constructors with logic around which one to use starts getting messy, but let's go ahead and remove the old constructor (or rather, change the signature of the existing one depending on how you look at it) since we don't need it anymore. Make possible to authenticate user when loading data to Cassandra with BulkRecordWriter. Key: CASSANDRA-4155 URL: https://issues.apache.org/jira/browse/CASSANDRA-4155 Project: Cassandra Issue Type: Improvement Reporter: Michał Michalski Priority: Minor Fix For: 1.1.1 Attachments: trunk-4155-nicer-version.txt, trunk-4155.txt I need to use the authorization feature (i.e. provided by SimpleAuthenticator and SimpleAuthority). The problem is that it's impossible now to pass the credentials (cassandra.output.keyspace.username and cassandra.output.keyspace.passwd) to org.apache.cassandra.hadoop.ConfigHelper because of no setters for these variables. Moreover, even if they could be passed, nothing will change because they are unused - ExternalClient class from org.apache.cassandra.hadoop.BulkRecordWriter is not making use of them; it's not even receiving them and no authorization is provided. The proposed improvement is to make it possible to authenticate user when loading data to Cassandra with BulkRecordWriter by adding appropriate setters to ConfigHelper and then passing credentials to ExternalClient class so it could use it for authorization request. I have created a patch for this which I attach. This improvement was made in the way that does not charm existing ExternalClient interface usage, but I think that it would be a bit nicer way to call the new constructor every time (optionally with username and password set to null) in this code and keeping the old one, instead of having and using two different constructors in two different cases in one method. However, it's my first patch for Cassandra, so I submit a less agressive one and I'm waiting for suggestions for to modify it :) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3909) Pig should handle wide rows
[ https://issues.apache.org/jira/browse/CASSANDRA-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3909: Attachment: 3909.txt CASSANDRA-3264 (and subsequently CASSANDRA-3883) added wide row support to hadoop, by returning one column in the row in every call. Pig, however, is fancy enough that it could handle a wide row in a bag, since bags spill to disk; it just needs the pagination to for transport since thrift doesn't stream. Also, if we returned what CFIF gave us, a user wanting to work within the row would need another costly M/R job to join the row back to its original state, so we essentially need to 'undo' the pagination and rebuild the row as a bag. This patch does that, with the caveat that you cannot access any indexes (and frankly if you have indexes on a wide row you're probably doing something wrong) since it's impossible for us to order the indexes correctly ahead of time in a wide row. Pig should handle wide rows --- Key: CASSANDRA-3909 URL: https://issues.apache.org/jira/browse/CASSANDRA-3909 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1.1 Attachments: 3909.txt Pig should be able to use the wide row support in CFIF. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3909) Pig should handle wide rows
[ https://issues.apache.org/jira/browse/CASSANDRA-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3909: Reviewer: xedin Pig should handle wide rows --- Key: CASSANDRA-3909 URL: https://issues.apache.org/jira/browse/CASSANDRA-3909 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1.1 Attachments: 3909.txt Pig should be able to use the wide row support in CFIF. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4155) Make possible to authenticate user when loading data to Cassandra with BulkRecordWriter.
[ https://issues.apache.org/jira/browse/CASSANDRA-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4155: Reviewer: brandon.williams Labels: (was: patch) Make possible to authenticate user when loading data to Cassandra with BulkRecordWriter. Key: CASSANDRA-4155 URL: https://issues.apache.org/jira/browse/CASSANDRA-4155 Project: Cassandra Issue Type: Improvement Reporter: Michał Michalski Priority: Minor Attachments: trunk-4155-nicer-version.txt, trunk-4155.txt I need to use the authorization feature (i.e. provided by SimpleAuthenticator and SimpleAuthority). The problem is that it's impossible now to pass the credentials (cassandra.output.keyspace.username and cassandra.output.keyspace.passwd) to org.apache.cassandra.hadoop.ConfigHelper because of no setters for these variables. Moreover, even if they could be passed, nothing will change because they are unused - ExternalClient class from org.apache.cassandra.hadoop.BulkRecordWriter is not making use of them; it's not even receiving them and no authorization is provided. The proposed improvement is to make it possible to authenticate user when loading data to Cassandra with BulkRecordWriter by adding appropriate setters to ConfigHelper and then passing credentials to ExternalClient class so it could use it for authorization request. I have created a patch for this which I attach. This improvement was made in the way that does not charm existing ExternalClient interface, but I think that it would be a bit nicer way to call the new constructor every time (optionally with username and password set to null) in this code and keeping the old one, instead of having and using two different constructors in two different cases in one method. However, it's my first patch for Cassandra, so I submit a less agressive one and I'm waiting for suggestions for to modify it :) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4146) Delete sstables on abnormal termination of sstableloader session
[ https://issues.apache.org/jira/browse/CASSANDRA-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4146: Reviewer: (was: jbellis) Priority: Minor (was: Major) Fix Version/s: (was: 1.1.0) 1.1.1 I don't think deleting the sstables is the right thing to do, since the loader didn't create those files, and they are potentially very expensive to recreate. However, I do understand that the loader not failing, nor being able to give any indication of failure is a problem, and now that we have CASSANDRA-4051 we can address that. Delete sstables on abnormal termination of sstableloader session Key: CASSANDRA-4146 URL: https://issues.apache.org/jira/browse/CASSANDRA-4146 Project: Cassandra Issue Type: Improvement Components: Tools Affects Versions: 1.0.9 Reporter: Manish Zope Priority: Minor Labels: sstableloader, tools Fix For: 1.1.1 Original Estimate: 48h Remaining Estimate: 48h There are three cases where we have observed the abnormal termination 1) In case of exception while loading. 2) User terminates the loading process. 3) If some node is down OR un-reachable then sstableloader get stucked.In this case user have to terminate the process in between. In case of abnormal termination, sstables (which are added in this session) remains as it is on the cluster.In case user starts the process all over again by fixing the exception, it results in duplication of the data till Major compaction is triggered. sstableloader can maintain the session while loading the sstables in cluster.So in case of abnormal termination sstableloader triggers the event that will delete the sstables loaded in that session. Also It would be great to have timeout in case of sstableloader.That can be kept configurable. So if sstableloader process got stucked for period longer than timeout, it can terminate itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4146) Delete sstables on abnormal termination of sstableloader session
[ https://issues.apache.org/jira/browse/CASSANDRA-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4146: Attachment: 4146.txt Patch to detect and report failures, as well as exit with a non-zero return code if this happens. Also cuts the FST attempts to 4, which causes about 28s of delay, instead of 8 which results in a ridiculous 8+ minutes. Delete sstables on abnormal termination of sstableloader session Key: CASSANDRA-4146 URL: https://issues.apache.org/jira/browse/CASSANDRA-4146 Project: Cassandra Issue Type: Improvement Components: Tools Affects Versions: 1.0.9 Reporter: Manish Zope Priority: Minor Labels: sstableloader, tools Fix For: 1.1.1 Attachments: 4146.txt Original Estimate: 48h Remaining Estimate: 48h There are three cases where we have observed the abnormal termination 1) In case of exception while loading. 2) User terminates the loading process. 3) If some node is down OR un-reachable then sstableloader get stucked.In this case user have to terminate the process in between. In case of abnormal termination, sstables (which are added in this session) remains as it is on the cluster.In case user starts the process all over again by fixing the exception, it results in duplication of the data till Major compaction is triggered. sstableloader can maintain the session while loading the sstables in cluster.So in case of abnormal termination sstableloader triggers the event that will delete the sstables loaded in that session. Also It would be great to have timeout in case of sstableloader.That can be kept configurable. So if sstableloader process got stucked for period longer than timeout, it can terminate itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4146) sstableloader should detect and report failures
[ https://issues.apache.org/jira/browse/CASSANDRA-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4146: Summary: sstableloader should detect and report failures (was: Delete sstables on abnormal termination of sstableloader session) sstableloader should detect and report failures --- Key: CASSANDRA-4146 URL: https://issues.apache.org/jira/browse/CASSANDRA-4146 Project: Cassandra Issue Type: Improvement Components: Tools Affects Versions: 1.0.9 Reporter: Manish Zope Priority: Minor Labels: sstableloader, tools Fix For: 1.1.1 Attachments: 4146.txt Original Estimate: 48h Remaining Estimate: 48h There are three cases where we have observed the abnormal termination 1) In case of exception while loading. 2) User terminates the loading process. 3) If some node is down OR un-reachable then sstableloader get stucked.In this case user have to terminate the process in between. In case of abnormal termination, sstables (which are added in this session) remains as it is on the cluster.In case user starts the process all over again by fixing the exception, it results in duplication of the data till Major compaction is triggered. sstableloader can maintain the session while loading the sstables in cluster.So in case of abnormal termination sstableloader triggers the event that will delete the sstables loaded in that session. Also It would be great to have timeout in case of sstableloader.That can be kept configurable. So if sstableloader process got stucked for period longer than timeout, it can terminate itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4045) BOF fails when some nodes are down
[ https://issues.apache.org/jira/browse/CASSANDRA-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4045: Attachment: 4045.txt Patch requires CASSANDRA-4146. It would be nice if we could support the notion of consistency level here, but it's not as easy as setting a ConsistencyLevel directive like other output formats, so instead we just allow defining a maximum number of hosts that are allowed to fail before the entire job fails, and default that to zero. BOF fails when some nodes are down -- Key: CASSANDRA-4045 URL: https://issues.apache.org/jira/browse/CASSANDRA-4045 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Labels: hadoop Fix For: 1.1.1 Attachments: 4045.txt As the summary says, we should allow jobs to complete when some targets are unavailable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4115) UNREACHABLE schema after decommissioning a non-seed node
[
https://issues.apache.org/jira/browse/CASSANDRA-4115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-4115:
Attachment: 4115.txt
The only way I can see this happening on 1.0 is if ring delay is set to
something lower than the gossip interval *2, otherwise it might not actually
send the 'left' message. That would result in the node being stuck in a
'leaving' and down state keeping it in the unreachable map.
For 1.1 and trunk, this is caused by CASSANDRA-3936, forcing a conviction when
the decom node shuts the gossiper down, after the others have already removed
it. Instead, markAlive and markDead should have a dead state check and thus
ignore it, since it's never valid to do either with a dead state.
UNREACHABLE schema after decommissioning a non-seed node
Key: CASSANDRA-4115
URL: https://issues.apache.org/jira/browse/CASSANDRA-4115
Project: Cassandra
Issue Type: Bug
Environment: ccm using the following unavailable_schema_test.py dtest.
Reporter: Tyler Patterson
Assignee: Brandon Williams
Priority: Minor
Attachments: 4115.txt
decommission a non-seed node, sleep 30 seconds, then use thrift to check the
schema. UNREACHABLE is listed:
{'75dc4c07-3c1a-3013-ad7d-11fb34208465': ['127.0.0.1'],
'UNREACHABLE': ['127.0.0.2']}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3722) Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.
[ https://issues.apache.org/jira/browse/CASSANDRA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3722: Attachment: 3722-v4.txt I think requests pending just isn't a good measure, especially for a multi-dc setup. Instead what we can do is penalize hosts based on the last time they replied, with a cap of the update interval. This way, if we aren't querying cross-dc, that dc is penalized, but within a reasonable limit such that if the local reads get bad we'll exceed the threshold and go cross-dc if we have to. This seems to respond much better to my torture test of force-suspending a JVM, but not long enough for the FD to kick in. Send Hints to Dynamic Snitch when Compaction or repair is going on for a node. -- Key: CASSANDRA-3722 URL: https://issues.apache.org/jira/browse/CASSANDRA-3722 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1.0 Reporter: Vijay Assignee: Vijay Priority: Minor Attachments: 0001-CASSANDRA-3722-A1-V2.patch, 0001-CASSANDRA-3722-A1.patch, 0001-CASSANDRA-3722-v3.patch, 0001-CASSANDRA-3723-A2-Patch.patch, 0001-Expose-SP-latencies-in-nodetool-proxyhistograms.txt, 3722-v4.txt Currently Dynamic snitch looks at the latency for figuring out which node will be better serving the requests, this works great but there is a part of the traffic sent to collect this data... There is also a window when Snitch doesn't know about some major event which are going to happen on the node (Node which is going to receive the data request). It would be great if we can send some sort hints to the Snitch so they can score based on known events causing higher latencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3722) Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.
[ https://issues.apache.org/jira/browse/CASSANDRA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3722: Attachment: 3722-v5.txt v5 does some minor clean up (REPORT renamed to SEVERITY mostly) and fixes the dsnitch test so it passes. Send Hints to Dynamic Snitch when Compaction or repair is going on for a node. -- Key: CASSANDRA-3722 URL: https://issues.apache.org/jira/browse/CASSANDRA-3722 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1.0 Reporter: Vijay Assignee: Vijay Priority: Minor Attachments: 0001-CASSANDRA-3722-A1-V2.patch, 0001-CASSANDRA-3722-A1.patch, 0001-CASSANDRA-3722-v3.patch, 0001-CASSANDRA-3723-A2-Patch.patch, 0001-Expose-SP-latencies-in-nodetool-proxyhistograms.txt, 3722-v4.txt, 3722-v5.txt Currently Dynamic snitch looks at the latency for figuring out which node will be better serving the requests, this works great but there is a part of the traffic sent to collect this data... There is also a window when Snitch doesn't know about some major event which are going to happen on the node (Node which is going to receive the data request). It would be great if we can send some sort hints to the Snitch so they can score based on known events causing higher latencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4101) Gossip should propagate MessagingService.version
[ https://issues.apache.org/jira/browse/CASSANDRA-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4101: Attachment: 4101.txt Gossip should propagate MessagingService.version Key: CASSANDRA-4101 URL: https://issues.apache.org/jira/browse/CASSANDRA-4101 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Priority: Critical Fix For: 1.1.0 Attachments: 4101.txt In CASSANDRA-4099 it's becoming apparent that it's time to fix our hacky versioning tricks we've used to remain backward-compatible. As a first step, let's communicate the version via gossip so we can eventually reason based on that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4101) Gossip should propagate MessagingService.version
[ https://issues.apache.org/jira/browse/CASSANDRA-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4101: Reviewer: vijay2...@yahoo.com Gossip should propagate MessagingService.version Key: CASSANDRA-4101 URL: https://issues.apache.org/jira/browse/CASSANDRA-4101 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Priority: Critical Fix For: 1.1.0 Attachments: 4101.txt In CASSANDRA-4099 it's becoming apparent that it's time to fix our hacky versioning tricks we've used to remain backward-compatible. As a first step, let's communicate the version via gossip so we can eventually reason based on that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4086) decom should shut thrift down
[ https://issues.apache.org/jira/browse/CASSANDRA-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4086: Attachment: 4086.txt Trivial patch to shut thrift down just before gossip/MS. decom should shut thrift down - Key: CASSANDRA-4086 URL: https://issues.apache.org/jira/browse/CASSANDRA-4086 Project: Cassandra Issue Type: Bug Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Fix For: 1.0.9, 1.1.0 Attachments: 4086.txt If you decom a node an then try to use it, you get nothing but timeouts. Instead let's just kill thrift so intelligent clients can move along. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3722) Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.
[ https://issues.apache.org/jira/browse/CASSANDRA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3722: Attachment: 0001-Expose-SP-latencies-in-nodetool-proxyhistograms.txt Here's a patch to make this a bit easier to test, it exposes SP recent latency histograms via nodetool. Send Hints to Dynamic Snitch when Compaction or repair is going on for a node. -- Key: CASSANDRA-3722 URL: https://issues.apache.org/jira/browse/CASSANDRA-3722 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1.0 Reporter: Vijay Assignee: Vijay Priority: Minor Attachments: 0001-CASSANDRA-3722-A1-V2.patch, 0001-CASSANDRA-3722-A1.patch, 0001-CASSANDRA-3723-A2-Patch.patch, 0001-Expose-SP-latencies-in-nodetool-proxyhistograms.txt Currently Dynamic snitch looks at the latency for figuring out which node will be better serving the requests, this works great but there is a part of the traffic sent to collect this data... There is also a window when Snitch doesn't know about some major event which are going to happen on the node (Node which is going to receive the data request). It would be great if we can send some sort hints to the Snitch so they can score based on known events causing higher latencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4086) decom should shut thrift down
[ https://issues.apache.org/jira/browse/CASSANDRA-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4086: Fix Version/s: 1.1.0 1.0.9 decom should shut thrift down - Key: CASSANDRA-4086 URL: https://issues.apache.org/jira/browse/CASSANDRA-4086 Project: Cassandra Issue Type: Bug Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.0.9, 1.1.0 If you decom a node an then try to use it, you get nothing but timeouts. Instead let's just kill thrift so intelligent clients can move along. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4067) Report lifetime compaction throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4067: Attachment: (was: 4067.txt) Report lifetime compaction throughput - Key: CASSANDRA-4067 URL: https://issues.apache.org/jira/browse/CASSANDRA-4067 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: Brandon Williams Priority: Trivial Labels: compaction Fix For: 1.1.0 Would be useful to be able to monitor total compaction throughput without having to poll frequently enough to make sure we get every CompactionInfo object. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4067) Report lifetime compaction throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4067: Attachment: 0002-Track-and-expose-total-compactions.txt 0001-Track-and-expose-lifetime-bytes-compacted.txt Updated patchset to also expose total compactions. Report lifetime compaction throughput - Key: CASSANDRA-4067 URL: https://issues.apache.org/jira/browse/CASSANDRA-4067 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: Brandon Williams Priority: Trivial Labels: compaction Fix For: 1.1.0 Attachments: 0001-Track-and-expose-lifetime-bytes-compacted.txt, 0002-Track-and-expose-total-compactions.txt Would be useful to be able to monitor total compaction throughput without having to poll frequently enough to make sure we get every CompactionInfo object. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4051) Stream sessions can only fail via the FailureDetector
[ https://issues.apache.org/jira/browse/CASSANDRA-4051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4051: Reviewer: yukim (was: slebresne) Stream sessions can only fail via the FailureDetector - Key: CASSANDRA-4051 URL: https://issues.apache.org/jira/browse/CASSANDRA-4051 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Labels: streaming Fix For: 1.1.0 Attachments: 4051.txt If for some reason, FileStreamTask itself fails more than the number of retry attempts but gossip continues to work, the stream session will never be closed. This is unlikely to happen in practice since it requires blocking the storage port from new connections but keeping the existing ones, however for the bulk loader this is especially problematic since it doesn't have access to a failure detector and thus no way of knowing if a session failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4067) Report lifetime compaction throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4067: Attachment: 4067.txt Patch to expose the lifetime bytes compacted. Report lifetime compaction throughput - Key: CASSANDRA-4067 URL: https://issues.apache.org/jira/browse/CASSANDRA-4067 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: Brandon Williams Priority: Trivial Labels: compaction Fix For: 1.1.0 Attachments: 4067.txt Would be useful to be able to monitor total compaction throughput without having to poll frequently enough to make sure we get every CompactionInfo object. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4067) Report lifetime compaction throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4067: Attachment: (was: 4067.txt) Report lifetime compaction throughput - Key: CASSANDRA-4067 URL: https://issues.apache.org/jira/browse/CASSANDRA-4067 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: Brandon Williams Priority: Trivial Labels: compaction Fix For: 1.1.0 Would be useful to be able to monitor total compaction throughput without having to poll frequently enough to make sure we get every CompactionInfo object. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4067) Report lifetime compaction throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4067: Attachment: 4067.txt Updated to include bytes completed up to the moment, instead of only showing completed compactions. Report lifetime compaction throughput - Key: CASSANDRA-4067 URL: https://issues.apache.org/jira/browse/CASSANDRA-4067 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: Brandon Williams Priority: Trivial Labels: compaction Fix For: 1.1.0 Attachments: 4067.txt Would be useful to be able to monitor total compaction throughput without having to poll frequently enough to make sure we get every CompactionInfo object. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4063) Expose nodetool cfhistograms for secondary index CFs
[
https://issues.apache.org/jira/browse/CASSANDRA-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-4063:
Attachment: 4063.txt
The reason wildcarding didn't work is because when you use them you have to
pass that to queryNames first, and then use the returned set of ObjectNames.
However, it's very nasty of JMX to allow you to pass wilcards in the ObjectName
and then do the wrong thing with it.
Expose nodetool cfhistograms for secondary index CFs
Key: CASSANDRA-4063
URL: https://issues.apache.org/jira/browse/CASSANDRA-4063
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Tyler Hobbs
Assignee: Brandon Williams
Priority: Minor
Labels: jmx
Attachments: 4063.txt
With the ObjectName that NodeProbe uses, the JMX query can only match mbeans
with type ColumnFamilies. Secondary index CFs have a type of
IndexColumnFamilies, so the query won't match them.
The [ObjectName
documentation|http://docs.oracle.com/javase/6/docs/api/javax/management/ObjectName.html]
indicates that you can use wildcards, which would be the perfect solution if
it actually worked. I'm not sure if it's some quoted vs non-quoted pattern
issue, or if it's particular to the {{newMBeanProxy()}} method, but I could
not get wildcards to match the secondary index CFs. Explicitly setting the
type field to IndexColumnFamilies did work.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4064) cfstats should include the cfId
[ https://issues.apache.org/jira/browse/CASSANDRA-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4064: Description: Specifically, when troubleshooting the dreaded UnserializableColumnFamilyException: Couldn't find cfId=1001 type of error (and the schema is in agreement), it would be really useful to easily see the cfIds so you know where the problem is (or is not) (was: Specifically, when troubleshooting the dreaded UnserializableColumnFamilyException: Couldn't find cfId=1001 type of error (and the schema is in agreement), it would be really useful to easily see the cfIds so you where the problem is (or is not)) cfstats should include the cfId --- Key: CASSANDRA-4064 URL: https://issues.apache.org/jira/browse/CASSANDRA-4064 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.0.9 Specifically, when troubleshooting the dreaded UnserializableColumnFamilyException: Couldn't find cfId=1001 type of error (and the schema is in agreement), it would be really useful to easily see the cfIds so you know where the problem is (or is not) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4064) cfstats should include the cfId
[ https://issues.apache.org/jira/browse/CASSANDRA-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4064: Priority: Minor (was: Major) This turned out to be CASSANDRA-3975, but still it could be handy if we have bugs here in the future. cfstats should include the cfId --- Key: CASSANDRA-4064 URL: https://issues.apache.org/jira/browse/CASSANDRA-4064 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Fix For: 1.0.9 Specifically, when troubleshooting the dreaded UnserializableColumnFamilyException: Couldn't find cfId=1001 type of error (and the schema is in agreement), it would be really useful to easily see the cfIds so you know where the problem is (or is not) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4051) Stream sessions can only fail via the FailureDetector
[ https://issues.apache.org/jira/browse/CASSANDRA-4051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4051: Attachment: 4051.txt Updated patch extracted as mentioned, doesn't change any streaming behavior but does provide a way to detect errors that CASSANDRA-3112 and CASSANDRA-4045 can build on. Stream sessions can only fail via the FailureDetector - Key: CASSANDRA-4051 URL: https://issues.apache.org/jira/browse/CASSANDRA-4051 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Labels: streaming Fix For: 1.1.0 Attachments: 4051.txt If for some reason, FileStreamTask itself fails more than the number of retry attempts but gossip continues to work, the stream session will never be closed. This is unlikely to happen in practice since it requires blocking the storage port from new connections but keeping the existing ones, however for the bulk loader this is especially problematic since it doesn't have access to a failure detector and thus no way of knowing if a session failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4051) Stream sessions can only fail via the FailureDetector
[ https://issues.apache.org/jira/browse/CASSANDRA-4051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4051: Reviewer: slebresne Labels: streaming (was: ) Stream sessions can only fail via the FailureDetector - Key: CASSANDRA-4051 URL: https://issues.apache.org/jira/browse/CASSANDRA-4051 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Labels: streaming Fix For: 1.1.0 Attachments: 4051.txt If for some reason, FileStreamTask itself fails more than the number of retry attempts but gossip continues to work, the stream session will never be closed. This is unlikely to happen in practice since it requires blocking the storage port from new connections but keeping the existing ones, however for the bulk loader this is especially problematic since it doesn't have access to a failure detector and thus no way of knowing if a session failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4051) Stream sessions can only fail via the FailureDetector
[ https://issues.apache.org/jira/browse/CASSANDRA-4051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4051: Affects Version/s: (was: 1.1.0) Stream sessions can only fail via the FailureDetector - Key: CASSANDRA-4051 URL: https://issues.apache.org/jira/browse/CASSANDRA-4051 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1.1 If for some reason, FileStreamTask itself fails more than the number of retry attempts but gossip continues to work, the stream session will never be closed. This is unlikely to happen in practice since it requires blocking the storage port from new connections but keeping the existing ones, however for the bulk loader this is especially problematic since it doesn't have access to a failure detector and thus no way of knowing if a session failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4051) Stream sessions can only fail via the FailureDetector
[ https://issues.apache.org/jira/browse/CASSANDRA-4051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4051: Fix Version/s: (was: 1.1.1) 1.1.0 Stream sessions can only fail via the FailureDetector - Key: CASSANDRA-4051 URL: https://issues.apache.org/jira/browse/CASSANDRA-4051 Project: Cassandra Issue Type: Bug Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1.0 If for some reason, FileStreamTask itself fails more than the number of retry attempts but gossip continues to work, the stream session will never be closed. This is unlikely to happen in practice since it requires blocking the storage port from new connections but keeping the existing ones, however for the bulk loader this is especially problematic since it doesn't have access to a failure detector and thus no way of knowing if a session failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3936) Gossip should have a 'goodbye' command to indicate shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3936: Attachment: 3936.txt Patch to send gossip shutdown messages when the gossiper is stopped. Gossip should have a 'goodbye' command to indicate shutdown --- Key: CASSANDRA-3936 URL: https://issues.apache.org/jira/browse/CASSANDRA-3936 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.2 Attachments: 3936.txt Cassandra is crash-only, however there are times when you _know_ you are taking the node down (rolling restarts, for instance) where it would be advantageous to instantly have the node marked down rather than wait on the FD. We could also improve the efficacy of the 'disablegossip' command this way as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3936) Gossip should have a 'goodbye' command to indicate shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3936: Attachment: 3936.txt Updated version obviates the need to quarantine after shutdown. Gossip should have a 'goodbye' command to indicate shutdown --- Key: CASSANDRA-3936 URL: https://issues.apache.org/jira/browse/CASSANDRA-3936 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.2 Attachments: 3936.txt Cassandra is crash-only, however there are times when you _know_ you are taking the node down (rolling restarts, for instance) where it would be advantageous to instantly have the node marked down rather than wait on the FD. We could also improve the efficacy of the 'disablegossip' command this way as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3936) Gossip should have a 'goodbye' command to indicate shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3936: Attachment: (was: 3936.txt) Gossip should have a 'goodbye' command to indicate shutdown --- Key: CASSANDRA-3936 URL: https://issues.apache.org/jira/browse/CASSANDRA-3936 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.2 Attachments: 3936.txt Cassandra is crash-only, however there are times when you _know_ you are taking the node down (rolling restarts, for instance) where it would be advantageous to instantly have the node marked down rather than wait on the FD. We could also improve the efficacy of the 'disablegossip' command this way as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3943) Too many small size sstables after loading data using sstableloader or BulkOutputFormat increases compaction time.
[
https://issues.apache.org/jira/browse/CASSANDRA-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3943:
Reviewer: (was: jbellis)
Assignee: (was: Brandon Williams)
Labels: bulkloader hadoop ponies sstableloader streaming tools (was:
bulkloader hadoop sstableloader streaming tools)
Too many small size sstables after loading data using sstableloader or
BulkOutputFormat increases compaction time.
--
Key: CASSANDRA-3943
URL: https://issues.apache.org/jira/browse/CASSANDRA-3943
Project: Cassandra
Issue Type: Wish
Components: Hadoop, Tools
Affects Versions: 0.8.2, 1.1.0
Reporter: Samarth Gahire
Priority: Minor
Labels: bulkloader, hadoop, ponies, sstableloader, streaming,
tools
Original Estimate: 168h
Remaining Estimate: 168h
When we create sstables using SimpleUnsortedWriter or BulkOutputFormat,the
size of sstables created is around the buffer size provided.
But After loading , sstables created in the cluster nodes are of size around
{code}( (sstable_size_before_loading) * replication_factor ) /
No_Of_Nodes_In_Cluster{code}
As the no of nodes in cluster goes increasing, size of each sstable loaded to
cassandra node decreases.Such small size sstables take too much time to
compact (minor compaction) as compare to relatively large size sstables.
One solution that we have tried is to increase the buffer size while
generating sstables.But as we increase the buffer size ,time taken to
generate sstables increases.Is there any solution to this in existing
versions or are you fixing this in future version?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3763) compactionstats throws ArithmeticException: / by zero
[ https://issues.apache.org/jira/browse/CASSANDRA-3763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3763: Priority: Trivial (was: Major) compactionstats throws ArithmeticException: / by zero - Key: CASSANDRA-3763 URL: https://issues.apache.org/jira/browse/CASSANDRA-3763 Project: Cassandra Issue Type: Bug Components: Core, Tools Affects Versions: 1.0.7, 1.0.8 Environment: debian linux - openvz kernel, oracle java 1.6.0.26 Reporter: Zenek Kraweznik Priority: Trivial compactionstats looks like this: # nodetool -h localhost compactionstats Exception in thread main java.lang.ArithmeticException: / by zero at org.apache.cassandra.db.compaction.LeveledManifest.getEstimatedTasks(LeveledManifest.java:435) at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getEstimatedRemainingTasks(LeveledCompactionStrategy.java:128) at org.apache.cassandra.db.compaction.CompactionManager.getPendingTasks(CompactionManager.java:1060) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208) at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:65) at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:216) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666) at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:638) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1404) at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360) at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:600) at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) at sun.rmi.transport.Transport$1.run(Transport.java:159) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:155) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) # nodetool is working fine in other actions: # nodetool -h localhost netstats Mode: NORMAL Not sending any streams. Not receiving any streams. Pool NameActive Pending Completed Commandsn/a 0 2 Responses n/a 0 1810 # -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3936) Gossip should have a 'goodbye' command to indicate shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3936: Attachment: 3936.txt Sleep for one gossip interval before announcing shutdown to ensure there isn't already a timed task launched that will make the node flap. Gossip should have a 'goodbye' command to indicate shutdown --- Key: CASSANDRA-3936 URL: https://issues.apache.org/jira/browse/CASSANDRA-3936 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.2 Attachments: 3936.txt Cassandra is crash-only, however there are times when you _know_ you are taking the node down (rolling restarts, for instance) where it would be advantageous to instantly have the node marked down rather than wait on the FD. We could also improve the efficacy of the 'disablegossip' command this way as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3936) Gossip should have a 'goodbye' command to indicate shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3936: Attachment: (was: 3936.txt) Gossip should have a 'goodbye' command to indicate shutdown --- Key: CASSANDRA-3936 URL: https://issues.apache.org/jira/browse/CASSANDRA-3936 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.2 Attachments: 3936.txt Cassandra is crash-only, however there are times when you _know_ you are taking the node down (rolling restarts, for instance) where it would be advantageous to instantly have the node marked down rather than wait on the FD. We could also improve the efficacy of the 'disablegossip' command this way as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4019) java.util.ConcurrentModificationException in Gossiper
[ https://issues.apache.org/jira/browse/CASSANDRA-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4019: Attachment: 4019.txt For 0.8/1.0, do the simplest thing and switch BDS from ArrayDeque to LinkedBlockingDeque, imitating the ALT behavior (which is well tested.) For trunk I'll do the aforementioned refator. java.util.ConcurrentModificationException in Gossiper - Key: CASSANDRA-4019 URL: https://issues.apache.org/jira/browse/CASSANDRA-4019 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.9 Reporter: Thibaut Priority: Minor Fix For: 0.8.11 Attachments: 4019.txt I have never seen this one before. Might be triggered by a race condition under heavy load. This error was triggered on 0.8.9 ERROR [GossipTasks:1] 2012-03-05 04:16:55,263 Gossiper.java (line 162) Gossip error java.util.ConcurrentModificationException at java.util.ArrayDeque$DeqIterator.next(ArrayDeque.java:605) at org.apache.cassandra.utils.AbstractStatsDeque.sum(AbstractStatsDeque.java:37) at org.apache.cassandra.utils.AbstractStatsDeque.mean(AbstractStatsDeque.java:60) at org.apache.cassandra.gms.ArrivalWindow.mean(FailureDetector.java:259) at org.apache.cassandra.gms.ArrivalWindow.phi(FailureDetector.java:282) at org.apache.cassandra.gms.FailureDetector.interpret(FailureDetector.java:155) at org.apache.cassandra.gms.Gossiper.doStatusCheck(Gossiper.java:538) at org.apache.cassandra.gms.Gossiper.access$700(Gossiper.java:57) at org.apache.cassandra.gms.Gossiper$GossipTask.run(Gossiper.java:157) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) INFO [GossipStage:1] 2012-03-05 04:16:55,263 Gossiper.java (line 737) Node /192.168.3.18 has restarted, now UP again INFO [GossipStage:1] 2012-03-05 04:16:55,264 Gossiper.java (line 705) InetAddress /192.168.3.18 is now UP -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4022) Compaction of hints can get stuck in a loop
[
https://issues.apache.org/jira/browse/CASSANDRA-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-4022:
Fix Version/s: (was: 1.1.0)
1.2
Assignee: Yuki Morishita (was: Brandon Williams)
Confirmed that this only happens with CASSANDRA-3442 applied.
Compaction of hints can get stuck in a loop
---
Key: CASSANDRA-4022
URL: https://issues.apache.org/jira/browse/CASSANDRA-4022
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Brandon Williams
Assignee: Yuki Morishita
Priority: Critical
Fix For: 1.2
Not exactly sure how I caused this as I was working on something else in
trunk, but:
{noformat}
INFO 17:41:35,682 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-339-Data.db')]
INFO 17:41:36,430 Compacted to
[/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-340-Data.db,].
4,637,160 to 4,637,160 (~100% of original) bytes
for 1 keys at 5.912220MB/s. Time: 748ms.
INFO 17:41:36,431 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-340-Data.db')]
INFO 17:41:37,238 Compacted to
[/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-341-Data.db,].
4,637,160 to 4,637,160 (~100% of original) bytes
for 1 keys at 5.479976MB/s. Time: 807ms.
INFO 17:41:37,239 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-341-Data.db')]
INFO 17:41:38,163 Compacted to
[/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-342-Data.db,].
4,637,160 to 4,637,160 (~100% of original) bytes
for 1 keys at 4.786083MB/s. Time: 924ms.
INFO 17:41:38,164 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-342-Data.db')]
INFO 17:41:39,014 GC for ParNew: 274 ms for 1 collections, 541261288 used;
max is 1024458752
INFO 17:41:39,151 Compacted to
[/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-343-Data.db,].
4,637,160 to 4,637,160 (~100% of original) bytes
for 1 keys at 4.485132MB/s. Time: 986ms.
INFO 17:41:39,151 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-343-Data.db')]
INFO 17:41:40,016 GC for ParNew: 308 ms for 1 collections, 585582200 used;
max is 1024458752
INFO 17:41:40,200 Compacted to
[/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-344-Data.db,].
4,637,160 to 4,637,160 (~100% of original) bytes
for 1 keys at 4.223821MB/s. Time: 1,047ms.
INFO 17:41:40,201 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-344-Data.db')]
INFO 17:41:41,017 GC for ParNew: 252 ms for 1 collections, 617877904 used;
max is 1024458752
INFO 17:41:41,178 Compacted to
[/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-345-Data.db,].
4,637,160 to 4,637,160 (~100% of original) bytes
for 1 keys at 4.526449MB/s. Time: 977ms.
INFO 17:41:41,179 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-345-Data.db')]
INFO 17:41:41,885 Compacted to
[/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-346-Data.db,].
4,637,160 to 4,637,160 (~100% of original) bytes
for 1 keys at 6.263938MB/s. Time: 706ms.
INFO 17:41:41,887 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-346-Data.db')]
INFO 17:41:42,617 Compacted to
[/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-347-Data.db,].
4,637,160 to 4,637,160 (~100% of original) bytes for 1 keys at
6.066311MB/s. Time: 729ms.
INFO 17:41:42,618 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-347-Data.db')]
INFO 17:41:43,376 Compacted to
[/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-348-Data.db,].
4,637,160 to 4,637,160 (~100% of original) bytes for 1 keys at
5.834222MB/s. Time: 758ms.
INFO 17:41:43,377 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-348-Data.db')]
INFO 17:41:44,307 Compacted to
[/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-349-Data.db,].
4,637,160 to 4,637,160 (~100% of original) bytes for 1 keys at
4.760323MB/s. Time: 929ms.
INFO 17:41:44,308 Compacting
[jira] [Updated] (CASSANDRA-4021) CFS.scrubDataDirectories tries to delete nonexistent orphans
[
https://issues.apache.org/jira/browse/CASSANDRA-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-4021:
Component/s: Core
Affects Version/s: 0.7 beta 2
Fix Version/s: 1.0.9
0.8.11
Assignee: Brandon Williams
CFS.scrubDataDirectories tries to delete nonexistent orphans
Key: CASSANDRA-4021
URL: https://issues.apache.org/jira/browse/CASSANDRA-4021
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7 beta 2
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
Fix For: 0.8.11, 1.0.9
The check only looks for a missing data file, then deletes all other
components, however it's possible for the data file and another component to
be missing, causing an error:
{noformat}
WARN 17:19:28,765 Removing orphans for
/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-24492:
[Index.db, Filter.db, Digest.sha1, Statistics.db, Data.db]
ERROR 17:19:28,766 Exception encountered during startup
java.lang.AssertionError: attempted to delete non-existing file
system-HintsColumnFamily-hd-24492-Index.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:49)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:44)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:357)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
at
org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:352)
at
org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:105)
java.lang.AssertionError: attempted to delete non-existing file
system-HintsColumnFamily-hd-24492-Index.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:49)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:44)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:357)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
at
org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:352)
at
org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:105)
Exception encountered during startup: attempted to delete non-existing file
system-HintsColumnFamily-hd-24492-Index.db
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4021) CFS.scrubDataDirectories tries to delete nonexistent orphans
[
https://issues.apache.org/jira/browse/CASSANDRA-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-4021:
Attachment: 4021.txt
Apparently this goes all the way back to CASSANDRA-1471 but occurs very rarely.
Patch to attempt deletion without caring about success.
CFS.scrubDataDirectories tries to delete nonexistent orphans
Key: CASSANDRA-4021
URL: https://issues.apache.org/jira/browse/CASSANDRA-4021
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7 beta 2
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
Fix For: 0.8.11, 1.0.9
Attachments: 4021.txt
The check only looks for a missing data file, then deletes all other
components, however it's possible for the data file and another component to
be missing, causing an error:
{noformat}
WARN 17:19:28,765 Removing orphans for
/var/lib/cassandra/data/system/HintsColumnFamily/system-HintsColumnFamily-hd-24492:
[Index.db, Filter.db, Digest.sha1, Statistics.db, Data.db]
ERROR 17:19:28,766 Exception encountered during startup
java.lang.AssertionError: attempted to delete non-existing file
system-HintsColumnFamily-hd-24492-Index.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:49)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:44)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:357)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
at
org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:352)
at
org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:105)
java.lang.AssertionError: attempted to delete non-existing file
system-HintsColumnFamily-hd-24492-Index.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:49)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:44)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:357)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
at
org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:352)
at
org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:105)
Exception encountered during startup: attempted to delete non-existing file
system-HintsColumnFamily-hd-24492-Index.db
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3991: Priority: Minor (was: Major) Investigate importance of jsvc in debian packages - Key: CASSANDRA-3991 URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Priority: Minor Fix For: 1.1.1 jsvc seems to be buggy at best. For instance, if you set a small heap like 128M it seems to completely ignore this and use as much memory as it wants. I don't know what this is buying us over launching /usr/bin/cassandra directly like the redhat scripts do, but I've seen multiple complaints about its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4026) EC2 snitch incorrectly reports regions
[ https://issues.apache.org/jira/browse/CASSANDRA-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-4026: Priority: Major (was: Critical) EC2 snitch incorrectly reports regions -- Key: CASSANDRA-4026 URL: https://issues.apache.org/jira/browse/CASSANDRA-4026 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.8 Environment: Ubuntu 10.10 64 bit Oracle Java 6 Reporter: Todd Nine Assignee: Vijay Currently the org.apache.cassandra.locator.Ec2Snitch reports us-west in both the oregon and the california data centers. This is incorrect, since they are different regions. California = us-west-1 Oregon = us-west-2 wget http://169.254.169.254/latest/meta-data/placement/availability-zone returns the value us-west-2a After parsing this returns DC = us-west Rack = 2a What it should return DC = us-west-2 Rack = a This makes it possible to use multi region when both regions are in the west coast. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3671) provide JMX counters for unavailables/timeouts for reads and writes
[ https://issues.apache.org/jira/browse/CASSANDRA-3671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3671: Fix Version/s: (was: 1.1.1) 1.1.0 provide JMX counters for unavailables/timeouts for reads and writes --- Key: CASSANDRA-3671 URL: https://issues.apache.org/jira/browse/CASSANDRA-3671 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Peter Schuller Assignee: Peter Schuller Priority: Minor Fix For: 1.1.0 Attachments: CASSANDRA-3671-trunk-coda-metrics-203-withjar.txt, CASSANDRA-3671-trunk-coda-metrics-v1.txt, CASSANDRA-3671-trunk-coda-metrics-v2.txt, CASSANDRA-3671-trunk-v2.txt, CASSANDRA-3671-trunk.txt, v1-0001-CASSANDRA-3671-trunk-coda-metrics-v2.txt.txt Attaching patch against trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3910) make phi_convict_threshold Float
[ https://issues.apache.org/jira/browse/CASSANDRA-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3910: Reviewer: brandon.williams make phi_convict_threshold Float Key: CASSANDRA-3910 URL: https://issues.apache.org/jira/browse/CASSANDRA-3910 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0.7 Reporter: Radim Kolar Assignee: Harish Doddi Attachments: CASSANDRA-3910-trunk.txt I would like to have phi_convict_threshold floating point number instead of integer. Value 8 is too low for me and value 9 is too high. With converting to floating point, it can be better fine tuned. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3671) provide JMX counters for unavailables/timeouts for reads and writes
[ https://issues.apache.org/jira/browse/CASSANDRA-3671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3671: Fix Version/s: (was: 1.1.0) 1.1.1 provide JMX counters for unavailables/timeouts for reads and writes --- Key: CASSANDRA-3671 URL: https://issues.apache.org/jira/browse/CASSANDRA-3671 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Peter Schuller Assignee: Peter Schuller Priority: Minor Fix For: 1.1.1 Attachments: CASSANDRA-3671-trunk-coda-metrics-203-withjar.txt, CASSANDRA-3671-trunk-coda-metrics-v1.txt, CASSANDRA-3671-trunk-coda-metrics-v2.txt, CASSANDRA-3671-trunk-v2.txt, CASSANDRA-3671-trunk.txt, v1-0001-CASSANDRA-3671-trunk-coda-metrics-v2.txt.txt Attaching patch against trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3991: Description: jsvc seems to be buggy at best. For instance, if you set a small heap like 128M it seems to completely ignore this and use as much memory as it wants. I don't know what this is buying us over launching /usr/bin/cassandra directly like the redhat scripts do, but I've seen multiple complaints about its memory usage. (was: jsvc seems to be buggy at best. For instance, if you set a small heap like 128M it seems to completely ignore this and use as much memory as it wants. I don't what this is buying us over launching /usr/bin/cassandra directly like the redhat scripts do, but I've seen multiple complaints about its memory usage.) Investigate importance of jsvc in debian packages - Key: CASSANDRA-3991 URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1.1 jsvc seems to be buggy at best. For instance, if you set a small heap like 128M it seems to completely ignore this and use as much memory as it wants. I don't know what this is buying us over launching /usr/bin/cassandra directly like the redhat scripts do, but I've seen multiple complaints about its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3986) Cli shouldn't call FBU.getBroadcastAddress needlessly
[
https://issues.apache.org/jira/browse/CASSANDRA-3986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3986:
Attachment: 3986.txt
Cli shouldn't call FBU.getBroadcastAddress needlessly
-
Key: CASSANDRA-3986
URL: https://issues.apache.org/jira/browse/CASSANDRA-3986
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.8
Reporter: Brandon Williams
Assignee: Brandon Williams
Fix For: 1.0.9, 1.1.0
Attachments: 3986.txt
The cli is calling this, which causes yaml to be loaded, but the broadcast
address isn't needed.
{code}
// adding default data center from SimpleSnitch
if (currentStrategyOptions == null ||
currentStrategyOptions.isEmpty())
{
SimpleSnitch snitch = new SimpleSnitch();
MapString, String options = new HashMapString, String();
options.put(snitch.getDatacenter(FBUtilities.getBroadcastAddress()), 1);
ksDef.setStrategy_options(options);
}
{code}
because SimpleSnitch always returns 'datacenter1'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3684) Composite Column Support for PIG
[ https://issues.apache.org/jira/browse/CASSANDRA-3684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3684: Reviewer: brandon.williams Fix Version/s: 1.1.0 1.0.9 Assignee: Janne Jalkanen (was: Brandon Williams) Composite Column Support for PIG Key: CASSANDRA-3684 URL: https://issues.apache.org/jira/browse/CASSANDRA-3684 Project: Cassandra Issue Type: Bug Reporter: Benjamin Coverston Assignee: Janne Jalkanen Fix For: 1.0.9, 1.1.0 Attachments: 3684-jalkanen-test-v2.txt, 3684-jalkanen-test.txt, 3684-jalkanen.txt It appears that some changes need to be made to support CompositeColumns. Right now if you try to load and use a column family that utilizes composite columns you get the following exception[1]. It appears to me that we need to modify the storage handler for Pig to support this scenario. [1] Backend error message - java.lang.RuntimeException: Unexpected data type -1 found in stream. at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:478) at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541) at org.apache.pig.data.BinInterSedes.writeBag(BinInterSedes.java:522) at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:361) at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541) at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:357) at org.apache.pig.data.BinSedesTuple.write(BinSedesTuple.java:57) at org.apache.pig.impl.io.PigNullableWritable.write(PigNullableWritable.java:123) at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:90) at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:77) at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1061) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691) at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:116) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:239) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:232) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:272) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) at org.apache.hadoop.mapred.Child.main(Child.java:266) Backend error message - java.lang.Throwable: Child Error at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) Caused by: java.io.IOException: Task process exit with nonzero status of 65. at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3973) Pig CounterColumnFamily support
[ https://issues.apache.org/jira/browse/CASSANDRA-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3973: Attachment: 3973-v2.txt v2 to nip counters directly in parseType so we can catch them in super columns too. Pig CounterColumnFamily support --- Key: CASSANDRA-3973 URL: https://issues.apache.org/jira/browse/CASSANDRA-3973 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.0.8 Environment: OSX 10.6.latest, Pig 0.9.2 Reporter: Janne Jalkanen Labels: counters, hadoop, pig Attachments: 3973-v2.txt, cassandrastorage.txt, testpatch.txt Included a patch to the current test script to demonstrate that CassandraStorage does not support counter columns, as well as a patch to fix the issue. The core reason is that CassandraStorage assumes that counters can be unpacked using CounterColumnType, when in fact the column value is already a Long. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3973) Pig CounterColumnFamily support
[ https://issues.apache.org/jira/browse/CASSANDRA-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3973: Reviewer: xedin Assignee: Brandon Williams Pig CounterColumnFamily support --- Key: CASSANDRA-3973 URL: https://issues.apache.org/jira/browse/CASSANDRA-3973 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.0.8 Environment: OSX 10.6.latest, Pig 0.9.2 Reporter: Janne Jalkanen Assignee: Brandon Williams Labels: counters, hadoop, pig Attachments: 3973-v2.txt, cassandrastorage.txt, testpatch.txt Included a patch to the current test script to demonstrate that CassandraStorage does not support counter columns, as well as a patch to fix the issue. The core reason is that CassandraStorage assumes that counters can be unpacked using CounterColumnType, when in fact the column value is already a Long. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3973) Pig CounterColumnFamily support
[ https://issues.apache.org/jira/browse/CASSANDRA-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3973: Reviewer: brandon.williams (was: xedin) Assignee: Janne Jalkanen (was: Brandon Williams) Pig CounterColumnFamily support --- Key: CASSANDRA-3973 URL: https://issues.apache.org/jira/browse/CASSANDRA-3973 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.0.8 Environment: OSX 10.6.latest, Pig 0.9.2 Reporter: Janne Jalkanen Assignee: Janne Jalkanen Labels: counters, hadoop, pig Attachments: 3973-v2.txt, cassandrastorage.txt, testpatch.txt, v3.txt Included a patch to the current test script to demonstrate that CassandraStorage does not support counter columns, as well as a patch to fix the issue. The core reason is that CassandraStorage assumes that counters can be unpacked using CounterColumnType, when in fact the column value is already a Long. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3763) compactionstats throws ArithmeticException: / by zero
[ https://issues.apache.org/jira/browse/CASSANDRA-3763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3763: Priority: Trivial (was: Major) compactionstats throws ArithmeticException: / by zero - Key: CASSANDRA-3763 URL: https://issues.apache.org/jira/browse/CASSANDRA-3763 Project: Cassandra Issue Type: Bug Components: Core, Tools Affects Versions: 1.0.7, 1.0.8 Environment: debian linux - openvz kernel, oracle java 1.6.0.26 Reporter: Zenek Kraweznik Priority: Trivial compactionstats looks like this: # nodetool -h localhost compactionstats Exception in thread main java.lang.ArithmeticException: / by zero at org.apache.cassandra.db.compaction.LeveledManifest.getEstimatedTasks(LeveledManifest.java:435) at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getEstimatedRemainingTasks(LeveledCompactionStrategy.java:128) at org.apache.cassandra.db.compaction.CompactionManager.getPendingTasks(CompactionManager.java:1060) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208) at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:65) at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:216) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666) at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:638) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1404) at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360) at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:600) at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) at sun.rmi.transport.Transport$1.run(Transport.java:159) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:155) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) # nodetool is working fine in other actions: # nodetool -h localhost netstats Mode: NORMAL Not sending any streams. Not receiving any streams. Pool NameActive Pending Completed Commandsn/a 0 2 Responses n/a 0 1810 # -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3972) HintedHandoff fails to deliver any hints
[ https://issues.apache.org/jira/browse/CASSANDRA-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3972: Attachment: 3972.txt That does indeed fix it. Patch to use the current time when filtering tombstones. HintedHandoff fails to deliver any hints Key: CASSANDRA-3972 URL: https://issues.apache.org/jira/browse/CASSANDRA-3972 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.0 Reporter: Brandon Williams Assignee: Brandon Williams Priority: Blocker Labels: hintedhandoff Fix For: 1.1.0 Attachments: 3972.txt Summary says it all. Whether in a memtable or sstable, no hints are delivered. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3972) HintedHandoff fails to deliver any hints
[ https://issues.apache.org/jira/browse/CASSANDRA-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3972: Reviewer: slebresne HintedHandoff fails to deliver any hints Key: CASSANDRA-3972 URL: https://issues.apache.org/jira/browse/CASSANDRA-3972 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.0 Reporter: Brandon Williams Assignee: Brandon Williams Priority: Blocker Labels: hintedhandoff Fix For: 1.1.0 Attachments: 3972.txt Summary says it all. Whether in a memtable or sstable, no hints are delivered. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3962) CassandraStorage can't cast fields from a CF correctly
[
https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3962:
Attachment: 0002-Compose-key-from-marshaller.txt
You're totally right. It wasn't a problem casting the key from Bytes, but
trying to use the one from U8. Patch to compose the key from the marshaller.
CassandraStorage can't cast fields from a CF correctly
--
Key: CASSANDRA-3962
URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.0.8
Environment: OSX 10.6.latest, Pig 0.9.2.
Reporter: Janne Jalkanen
Assignee: Brandon Williams
Labels: hadoop, pig
Attachments: 0001-Add-LoadCaster-to-CassandraStorage.txt,
0002-Compose-key-from-marshaller.txt, test.cli, test.pig
Included scripts demonstrate the problem. Regardless of whether the key is
cast as a chararray or not, the Pig scripts fail with
{code}
java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be
cast to java.lang.String
at
org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3962) CassandraStorage can't cast fields from a CF correctly
[
https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3962:
Reviewer: xedin
Fix Version/s: 1.1.0
1.0.9
CassandraStorage can't cast fields from a CF correctly
--
Key: CASSANDRA-3962
URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.0.8
Environment: OSX 10.6.latest, Pig 0.9.2.
Reporter: Janne Jalkanen
Assignee: Brandon Williams
Labels: hadoop, pig
Fix For: 1.0.9, 1.1.0
Attachments: 0001-Add-LoadCaster-to-CassandraStorage.txt,
0002-Compose-key-from-marshaller.txt, test.cli, test.pig
Included scripts demonstrate the problem. Regardless of whether the key is
cast as a chararray or not, the Pig scripts fail with
{code}
java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be
cast to java.lang.String
at
org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3962) CassandraStorage can't cast fields from a CF correctly
[
https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3962:
Attachment: 0001-Add-LoadCaster-to-CassandraStorage.txt
Patch to add a LoadCaster. It does get used and converts the byte[] to String,
but the join still fails with the same error :(
CassandraStorage can't cast fields from a CF correctly
--
Key: CASSANDRA-3962
URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.0.8
Environment: OSX 10.6.latest, Pig 0.9.2.
Reporter: Janne Jalkanen
Assignee: Brandon Williams
Labels: hadoop, pig
Attachments: 0001-Add-LoadCaster-to-CassandraStorage.txt, test.cli,
test.pig
Included scripts demonstrate the problem. Regardless of whether the key is
cast as a chararray or not, the Pig scripts fail with
{code}
java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be
cast to java.lang.String
at
org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3955) HintedHandoff won't compact a single sstable, resulting in repeated log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3955: Priority: Minor (was: Major) HintedHandoff won't compact a single sstable, resulting in repeated log messages Key: CASSANDRA-3955 URL: https://issues.apache.org/jira/browse/CASSANDRA-3955 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.7 Reporter: Brandon Williams Priority: Minor Fix For: 1.0.9 First introduced by CASSANDRA-3554, and then mostly solved in CASSANDRA-3733, there is still one special case where the HH log message will repeat every 10 mins for 0 rows: when there have previously been hints delivered to the node, but now only a single sstable exists. Because the we refused to compact a single sstable, and it contains tombstones for the hints, the message repeats. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3955) HintedHandoff won't compact a single sstable, resulting in repeated log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3955: Attachment: 3955.txt No need to special case, instead we'll just always force a user defined compaction. HintedHandoff won't compact a single sstable, resulting in repeated log messages Key: CASSANDRA-3955 URL: https://issues.apache.org/jira/browse/CASSANDRA-3955 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.7 Reporter: Brandon Williams Priority: Minor Fix For: 1.0.9 Attachments: 3955.txt First introduced by CASSANDRA-3554, and then mostly solved in CASSANDRA-3733, there is still one special case where the HH log message will repeat every 10 mins for 0 rows: when there have previously been hints delivered to the node, but now only a single sstable exists. Because the we refused to compact a single sstable, and it contains tombstones for the hints, the message repeats. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3943) Too many small size sstables after loading data using sstableloader or BulkOutputFormat increases compaction time.
[
https://issues.apache.org/jira/browse/CASSANDRA-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3943:
Fix Version/s: (was: 1.1.0)
Issue Type: Task (was: Improvement)
Too many small size sstables after loading data using sstableloader or
BulkOutputFormat increases compaction time.
--
Key: CASSANDRA-3943
URL: https://issues.apache.org/jira/browse/CASSANDRA-3943
Project: Cassandra
Issue Type: Task
Components: Hadoop, Tools
Affects Versions: 0.8.2, 1.1.0
Reporter: Samarth Gahire
Assignee: Brandon Williams
Priority: Minor
Labels: bulkloader, hadoop, sstableloader, streaming, tools
Original Estimate: 168h
Remaining Estimate: 168h
When we create sstables using SimpleUnsortedWriter or BulkOutputFormat,the
size of sstables created is around the buffer size provided.
But After loading , sstables created in the cluster nodes are of size around
{code}( (sstable_size_before_loading) * replication_factor ) /
No_Of_Nodes_In_Cluster{code}
As the no of nodes in cluster goes increasing, size of each sstable loaded to
cassandra node decreases.Such small size sstables take too much time to
compact (minor compaction) as compare to relatively large size sstables.
One solution that we have tried is to increase the buffer size while
generating sstables.But as we increase the buffer size ,time taken to
generate sstables increases.Is there any solution to this in existing
versions or are you fixing this in future version?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3943) Too many small size sstables after loading data using sstableloader or BulkOutputFormat increases compaction time.
[
https://issues.apache.org/jira/browse/CASSANDRA-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3943:
Issue Type: Wish (was: Task)
Too many small size sstables after loading data using sstableloader or
BulkOutputFormat increases compaction time.
--
Key: CASSANDRA-3943
URL: https://issues.apache.org/jira/browse/CASSANDRA-3943
Project: Cassandra
Issue Type: Wish
Components: Hadoop, Tools
Affects Versions: 0.8.2, 1.1.0
Reporter: Samarth Gahire
Assignee: Brandon Williams
Priority: Minor
Labels: bulkloader, hadoop, sstableloader, streaming, tools
Original Estimate: 168h
Remaining Estimate: 168h
When we create sstables using SimpleUnsortedWriter or BulkOutputFormat,the
size of sstables created is around the buffer size provided.
But After loading , sstables created in the cluster nodes are of size around
{code}( (sstable_size_before_loading) * replication_factor ) /
No_Of_Nodes_In_Cluster{code}
As the no of nodes in cluster goes increasing, size of each sstable loaded to
cassandra node decreases.Such small size sstables take too much time to
compact (minor compaction) as compare to relatively large size sstables.
One solution that we have tried is to increase the buffer size while
generating sstables.But as we increase the buffer size ,time taken to
generate sstables increases.Is there any solution to this in existing
versions or are you fixing this in future version?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3931) gossipers notion of schema differs from reality as reported by the nodes in question
[
https://issues.apache.org/jira/browse/CASSANDRA-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3931:
Attachment: 3931.txt
Patch to update gossip when merging remote versions.
gossipers notion of schema differs from reality as reported by the nodes in
question
Key: CASSANDRA-3931
URL: https://issues.apache.org/jira/browse/CASSANDRA-3931
Project: Cassandra
Issue Type: Bug
Reporter: Peter Schuller
Assignee: Peter Schuller
Fix For: 1.1.0
Attachments: 3931.txt
On a 1.1 cluster we happened to notice that {{nodetool gossipinfo | grep
SCHEMA}} reported disagreement:
{code}
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:b0d7bab7-c13c-37d9-9adb-8ab8a5b7215d
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:bcdbd318-82df-3518-89e3-6b72227b3f66
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:bcdbd318-82df-3518-89e3-6b72227b3f66
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
{code}
However, the result of a thrift {{describe_ring}} on the cluster claims they
all agree and that {{b0d7bab7-c13c-37d9-9adb-8ab8a5b7215d}} is the schema
they have.
The schemas seem to actually propagate; e.g. dropping a keyspace actually
drops the keyspace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3931) gossipers notion of schema differs from reality as reported by the nodes in question
[
https://issues.apache.org/jira/browse/CASSANDRA-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3931:
Reviewer: xedin
gossipers notion of schema differs from reality as reported by the nodes in
question
Key: CASSANDRA-3931
URL: https://issues.apache.org/jira/browse/CASSANDRA-3931
Project: Cassandra
Issue Type: Bug
Reporter: Peter Schuller
Assignee: Brandon Williams
Fix For: 1.1.0
Attachments: 3931.txt
On a 1.1 cluster we happened to notice that {{nodetool gossipinfo | grep
SCHEMA}} reported disagreement:
{code}
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:b0d7bab7-c13c-37d9-9adb-8ab8a5b7215d
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:bcdbd318-82df-3518-89e3-6b72227b3f66
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:bcdbd318-82df-3518-89e3-6b72227b3f66
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
{code}
However, the result of a thrift {{describe_ring}} on the cluster claims they
all agree and that {{b0d7bab7-c13c-37d9-9adb-8ab8a5b7215d}} is the schema
they have.
The schemas seem to actually propagate; e.g. dropping a keyspace actually
drops the keyspace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3859) Add Progress Reporting to Cassandra OutputFormats
[ https://issues.apache.org/jira/browse/CASSANDRA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3859: Attachment: (was: 0001-add-progress-reporting-to-BOF.txt) Add Progress Reporting to Cassandra OutputFormats - Key: CASSANDRA-3859 URL: https://issues.apache.org/jira/browse/CASSANDRA-3859 Project: Cassandra Issue Type: Improvement Components: Hadoop, Tools Affects Versions: 1.1.0 Reporter: Samarth Gahire Assignee: Brandon Williams Priority: Minor Labels: bulkloader, hadoop, mapreduce, sstableloader Fix For: 1.1.0 Attachments: 0002-Add-progress-to-CFOF.txt Original Estimate: 48h Remaining Estimate: 48h When we are using the BulkOutputFormat to load the data to cassandra. We should use the progress reporting to Hadoop Job within Sstable loader because while loading the data for particular task if streaming is taking more time and progress is not reported to Job it may kill the task with timeout exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3859) Add Progress Reporting to Cassandra OutputFormats
[ https://issues.apache.org/jira/browse/CASSANDRA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3859: Attachment: 0001-add-progress-reporting-to-BOF.txt You're totally right, I'm not sure what happened there since I obviously allocated a variable for the future like I was going to do the right thing, and then just... didn't. Updated 0001 loops while checking the future. Add Progress Reporting to Cassandra OutputFormats - Key: CASSANDRA-3859 URL: https://issues.apache.org/jira/browse/CASSANDRA-3859 Project: Cassandra Issue Type: Improvement Components: Hadoop, Tools Affects Versions: 1.1.0 Reporter: Samarth Gahire Assignee: Brandon Williams Priority: Minor Labels: bulkloader, hadoop, mapreduce, sstableloader Fix For: 1.1.0 Attachments: 0001-add-progress-reporting-to-BOF.txt, 0002-Add-progress-to-CFOF.txt Original Estimate: 48h Remaining Estimate: 48h When we are using the BulkOutputFormat to load the data to cassandra. We should use the progress reporting to Hadoop Job within Sstable loader because while loading the data for particular task if streaming is taking more time and progress is not reported to Job it may kill the task with timeout exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3815) Alllow compression setting adjustment via JMX
[ https://issues.apache.org/jira/browse/CASSANDRA-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3815: Attachment: 3815.txt Alllow compression setting adjustment via JMX - Key: CASSANDRA-3815 URL: https://issues.apache.org/jira/browse/CASSANDRA-3815 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1.1 Attachments: 3815.txt As the title says, let's allow enabling/disabling/setting chunk size via JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3815) Alllow compression setting adjustment via JMX
[ https://issues.apache.org/jira/browse/CASSANDRA-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3815: Reviewer: xedin Fix Version/s: (was: 1.1.1) 1.1.0 Alllow compression setting adjustment via JMX - Key: CASSANDRA-3815 URL: https://issues.apache.org/jira/browse/CASSANDRA-3815 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.1.0 Attachments: 3815.txt As the title says, let's allow enabling/disabling/setting chunk size via JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3913) Incorrect InetAddress equality test
[ https://issues.apache.org/jira/browse/CASSANDRA-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3913: Attachment: 3913.txt Incorrect InetAddress equality test --- Key: CASSANDRA-3913 URL: https://issues.apache.org/jira/browse/CASSANDRA-3913 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.9, 1.0.6 Reporter: Brandon Williams Assignee: Brandon Williams Priority: Trivial Fix For: 1.0.8 Attachments: 3913.txt CASSANDRA-3485 introduced some InetAddress checks using == instead of .equals. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3913) Incorrect InetAddress equality test
[ https://issues.apache.org/jira/browse/CASSANDRA-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3913: Fix Version/s: 0.8.11 Incorrect InetAddress equality test --- Key: CASSANDRA-3913 URL: https://issues.apache.org/jira/browse/CASSANDRA-3913 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.9, 1.0.6 Reporter: Brandon Williams Assignee: Brandon Williams Priority: Trivial Fix For: 0.8.11, 1.0.8 Attachments: 3913.txt CASSANDRA-3485 introduced some InetAddress checks using == instead of .equals. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
[
https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3740:
Reviewer: lenn0x (was: jbellis)
While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
--
Key: CASSANDRA-3740
URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.0
Reporter: Samarth Gahire
Assignee: Brandon Williams
Labels: cassandra, hadoop, mapreduce
Fix For: 1.1.0
Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt,
0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt,
0005-BWR-uses-any-if.txt
I am trying to use BulkOutputFormat to stream the data from map of Hadoop
job. I have set the cassandra related configuration using ConfigHelper ,Also
have looked into Cassandra code seems Cassandra has taken care that it should
not look for the cassandra.yaml file.
But still when I run the job i get the following error:
{
12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing
the arguments. Applications should implement Tool for the same.
12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
12/01/13 11:30:23 INFO mapred.JobClient: Task Id :
attempt_201201130910_0015_m_00_0, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
attempt_201201130910_0015_m_00_0: Cannot locate cassandra.yaml
attempt_201201130910_0015_m_00_0: Fatal configuration error; unable to
start server.
}
Also let me know how can i make this cassandra.yaml file available to Hadoop
mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3906) BulkRecordWriter throws NPE for counter columns
[ https://issues.apache.org/jira/browse/CASSANDRA-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3906: Attachment: 3906-v2.txt This patch doesn't look quite right, at the least the removal of the newSuperColumn call can't be right. Attaching v2. BulkRecordWriter throws NPE for counter columns --- Key: CASSANDRA-3906 URL: https://issues.apache.org/jira/browse/CASSANDRA-3906 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.0 Reporter: Chris Goffinet Assignee: Chris Goffinet Fix For: 1.1.0 Attachments: 0001-Fixed-BulkRecordWriter-to-not-throw-an-NPE-for-count.patch, 3906-v2.txt Using BulkRecordWriter, fails with counters due to an NPE (we used column instead of counter_column). I also noticed this broke for super columns too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3883) CFIF WideRowIterator only returns batch size columns
[ https://issues.apache.org/jira/browse/CASSANDRA-3883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3883: Assignee: (was: Brandon Williams) CFIF WideRowIterator only returns batch size columns Key: CASSANDRA-3883 URL: https://issues.apache.org/jira/browse/CASSANDRA-3883 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.1.0 Reporter: Brandon Williams Fix For: 1.1.0 Attachments: 3883-v1.txt Most evident with the word count, where there are 1250 'word1' items in two rows (1000 in one, 250 in another) and it counts 198 with the batch size set to 99. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3628) Make Pig/CassandraStorage delete functionality disabled by default and configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3628: Attachment: 3628-v2.txt v2 throws an error when a delete attempt is encountered but not explicitly enabled. Make Pig/CassandraStorage delete functionality disabled by default and configurable --- Key: CASSANDRA-3628 URL: https://issues.apache.org/jira/browse/CASSANDRA-3628 Project: Cassandra Issue Type: Task Reporter: Jeremy Hanna Assignee: Brandon Williams Labels: pig Fix For: 1.0.8 Attachments: 3628-v2.txt, 3628.txt Right now, there is a way to delete column with the CassandraStorage loadstorefunc. In practice it is a bad idea to have that enabled by default. A scenario: do an outer join and you don't have a value for something and then you write out to cassandra all of the attributes of that relation. You've just inadvertently deleted a column for all the rows that didn't have that value as a result of the outer join. It can be argued that you want to be careful with how you project after the join. However, I would think disabling by default and having a configurable property to enable it for the instances when you explicitly want to use it is the right plan. Fwiw, we had a bug in one of our scripts that did exactly as described above. It's good to fix the bug. It's bad to implicitly delete data. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3628) Make Pig/CassandraStorage delete functionality disabled by default and configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3628: Reviewer: xedin (was: brandon.williams) Make Pig/CassandraStorage delete functionality disabled by default and configurable --- Key: CASSANDRA-3628 URL: https://issues.apache.org/jira/browse/CASSANDRA-3628 Project: Cassandra Issue Type: Task Reporter: Jeremy Hanna Assignee: Brandon Williams Labels: pig Fix For: 1.0.8 Attachments: 3628-v2.txt, 3628.txt Right now, there is a way to delete column with the CassandraStorage loadstorefunc. In practice it is a bad idea to have that enabled by default. A scenario: do an outer join and you don't have a value for something and then you write out to cassandra all of the attributes of that relation. You've just inadvertently deleted a column for all the rows that didn't have that value as a result of the outer join. It can be argued that you want to be careful with how you project after the join. However, I would think disabling by default and having a configurable property to enable it for the instances when you explicitly want to use it is the right plan. Fwiw, we had a bug in one of our scripts that did exactly as described above. It's good to fix the bug. It's bad to implicitly delete data. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3883) CFIF WideRowIterator only returns batch size columns
[ https://issues.apache.org/jira/browse/CASSANDRA-3883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3883: Attachment: 3883-v1.txt v1 isn't perfect but it's a start; if the batch starts on a wide row, we reuse the token and iterate until we're done. Unfortunately if we don't start on one, I'm not sure if there's a way to detect that we're in a wide row without making an extra rpc against the last row seen every time. CFIF WideRowIterator only returns batch size columns Key: CASSANDRA-3883 URL: https://issues.apache.org/jira/browse/CASSANDRA-3883 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.1.0 Reporter: Brandon Williams Fix For: 1.1.0 Attachments: 3883-v1.txt Most evident with the word count, where there are 1250 'word1' items in two rows (1000 in one, 250 in another) and it counts 198 with the batch size set to 99. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3371) Cassandra inferred schema and actual data don't match
[
https://issues.apache.org/jira/browse/CASSANDRA-3371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3371:
Attachment: smoke_test.txt
3371-v6.txt
v6 is rebased and contains minor cleanups, smoke_test contains a file to be
replayed by the cli and a pig script to exercise loading/storing every
cassandra type.
Cassandra inferred schema and actual data don't match
-
Key: CASSANDRA-3371
URL: https://issues.apache.org/jira/browse/CASSANDRA-3371
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 0.8.7
Reporter: Pete Warden
Assignee: Brandon Williams
Attachments: 0001-Rework-pig-schema.txt,
0002-Output-support-to-match-input.txt, 3371-v2.txt, 3371-v3.txt,
3371-v4.txt, 3371-v5-rebased.txt, 3371-v5.txt, 3371-v6.txt, pig.diff,
smoke_test.txt
It's looking like there may be a mismatch between the schema that's being
reported by the latest CassandraStorage.java, and the data that's actually
returned. Here's an example:
rows = LOAD 'cassandra://Frap/PhotoVotes' USING CassandraStorage();
DESCRIBE rows;
rows: {key: chararray,columns: {(name: chararray,value:
bytearray,photo_owner: chararray,value_photo_owner: bytearray,pid:
chararray,value_pid: bytearray,matched_string:
chararray,value_matched_string: bytearray,src_big: chararray,value_src_big:
bytearray,time: chararray,value_time: bytearray,vote_type:
chararray,value_vote_type: bytearray,voter: chararray,value_voter:
bytearray)}}
DUMP rows;
(691831038_1317937188.48955,{(photo_owner,1596090180),(pid,6855155124568798560),(matched_string,),(src_big,),(time,Thu
Oct 06 14:39:48 -0700 2011),(vote_type,album_dislike),(voter,691831038)})
getSchema() is reporting the columns as an inner bag of tuples, each of which
contains 16 values. In fact, getNext() seems to return an inner bag
containing 7 tuples, each of which contains two values.
It appears that things got out of sync with this change:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java?r1=1177083r2=1177082pathrev=1177083
See more discussion at:
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/pig-cassandra-problem-quot-Incompatible-field-schema-quot-error-tc6882703.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
[
https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3740:
Attachment: (was: 0004-update-BOF-for-new-dir-layout.txt)
While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
--
Key: CASSANDRA-3740
URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.0
Reporter: Samarth Gahire
Assignee: Brandon Williams
Labels: cassandra, hadoop, mapreduce
Fix For: 1.1.0
Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt,
0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt,
0005-BWR-uses-any-if.txt
I am trying to use BulkOutputFormat to stream the data from map of Hadoop
job. I have set the cassandra related configuration using ConfigHelper ,Also
have looked into Cassandra code seems Cassandra has taken care that it should
not look for the cassandra.yaml file.
But still when I run the job i get the following error:
{
12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing
the arguments. Applications should implement Tool for the same.
12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
12/01/13 11:30:23 INFO mapred.JobClient: Task Id :
attempt_201201130910_0015_m_00_0, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
attempt_201201130910_0015_m_00_0: Cannot locate cassandra.yaml
attempt_201201130910_0015_m_00_0: Fatal configuration error; unable to
start server.
}
Also let me know how can i make this cassandra.yaml file available to Hadoop
mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3906) BulkRecordWriter throws NPE for counter columns
[ https://issues.apache.org/jira/browse/CASSANDRA-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3906: Reviewer: brandon.williams BulkRecordWriter throws NPE for counter columns --- Key: CASSANDRA-3906 URL: https://issues.apache.org/jira/browse/CASSANDRA-3906 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.0 Reporter: Chris Goffinet Assignee: Chris Goffinet Fix For: 1.1.0 Attachments: 0001-Fixed-BulkRecordWriter-to-not-throw-an-NPE-for-count.patch Using BulkRecordWriter, fails with counters due to an NPE (we used column instead of counter_column). I also noticed this broke for super columns too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3886) Pig can't store some types after loading them
[
https://issues.apache.org/jira/browse/CASSANDRA-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3886:
Attachment: 3886.txt
Patch to convert all possible types. Includes UUID, even though setTupleValue
should make it impossible for a UUID to get here, just in case.
Pig can't store some types after loading them
-
Key: CASSANDRA-3886
URL: https://issues.apache.org/jira/browse/CASSANDRA-3886
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 0.8.7
Reporter: Brandon Williams
Assignee: Brandon Williams
Fix For: 1.0.8
Attachments: 3886.txt
In CASSANDRA-2810, we removed the decompose methods in putNext instead
relying on objToBB, however it cannot sufficiently handle all types. For
instance, if longs are loaded and then an attempt to store them is made, this
causes a cast exception: java.io.IOException: java.io.IOException:
java.lang.ClassCastException: java.lang.Long cannot be cast to
org.apache.pig.data.DataByteArray Output must be (key, {(column,value)...})
for ColumnFamily or (key, {supercolumn:{(column,value)...}...}) for
SuperColumnFamily
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3886) Pig can't store some types after loading them
[
https://issues.apache.org/jira/browse/CASSANDRA-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3886:
Attachment: (was: 3886.txt)
Pig can't store some types after loading them
-
Key: CASSANDRA-3886
URL: https://issues.apache.org/jira/browse/CASSANDRA-3886
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 0.8.7
Reporter: Brandon Williams
Assignee: Brandon Williams
Fix For: 1.0.8
In CASSANDRA-2810, we removed the decompose methods in putNext instead
relying on objToBB, however it cannot sufficiently handle all types. For
instance, if longs are loaded and then an attempt to store them is made, this
causes a cast exception: java.io.IOException: java.io.IOException:
java.lang.ClassCastException: java.lang.Long cannot be cast to
org.apache.pig.data.DataByteArray Output must be (key, {(column,value)...})
for ColumnFamily or (key, {supercolumn:{(column,value)...}...}) for
SuperColumnFamily
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3886) Pig can't store some types after loading them
[
https://issues.apache.org/jira/browse/CASSANDRA-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3886:
Attachment: 3886.txt
Pig can't store some types after loading them
-
Key: CASSANDRA-3886
URL: https://issues.apache.org/jira/browse/CASSANDRA-3886
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 0.8.7
Reporter: Brandon Williams
Assignee: Brandon Williams
Fix For: 1.0.8
Attachments: 3886.txt
In CASSANDRA-2810, we removed the decompose methods in putNext instead
relying on objToBB, however it cannot sufficiently handle all types. For
instance, if longs are loaded and then an attempt to store them is made, this
causes a cast exception: java.io.IOException: java.io.IOException:
java.lang.ClassCastException: java.lang.Long cannot be cast to
org.apache.pig.data.DataByteArray Output must be (key, {(column,value)...})
for ColumnFamily or (key, {supercolumn:{(column,value)...}...}) for
SuperColumnFamily
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3859) Add Progress Reporting to Cassandra OutputFormats
[ https://issues.apache.org/jira/browse/CASSANDRA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3859: Fix Version/s: (was: 1.1.1) 1.1 Add Progress Reporting to Cassandra OutputFormats - Key: CASSANDRA-3859 URL: https://issues.apache.org/jira/browse/CASSANDRA-3859 Project: Cassandra Issue Type: Improvement Components: Hadoop, Tools Affects Versions: 1.1 Reporter: Samarth Gahire Assignee: Brandon Williams Priority: Minor Labels: bulkloader, hadoop, mapreduce, sstableloader Fix For: 1.1 Attachments: 0001-add-progress-reporting-to-BOF.txt, 0002-Add-progress-to-CFOF.txt Original Estimate: 48h Remaining Estimate: 48h When we are using the BulkOutputFormat to load the data to cassandra. We should use the progress reporting to Hadoop Job within Sstable loader because while loading the data for particular task if streaming is taking more time and progress is not reported to Job it may kill the task with timeout exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3859) Add Progress Reporting to Cassandra OutputFormats
[ https://issues.apache.org/jira/browse/CASSANDRA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3859: Attachment: (was: 0001-add-progress-reporting-to-BOF.txt) Add Progress Reporting to Cassandra OutputFormats - Key: CASSANDRA-3859 URL: https://issues.apache.org/jira/browse/CASSANDRA-3859 Project: Cassandra Issue Type: Improvement Components: Hadoop, Tools Affects Versions: 1.1 Reporter: Samarth Gahire Assignee: Brandon Williams Priority: Minor Labels: bulkloader, hadoop, mapreduce, sstableloader Fix For: 1.1 Attachments: 0002-Add-progress-to-CFOF.txt Original Estimate: 48h Remaining Estimate: 48h When we are using the BulkOutputFormat to load the data to cassandra. We should use the progress reporting to Hadoop Job within Sstable loader because while loading the data for particular task if streaming is taking more time and progress is not reported to Job it may kill the task with timeout exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3859) Add Progress Reporting to Cassandra OutputFormats
[ https://issues.apache.org/jira/browse/CASSANDRA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3859: Attachment: 0001-add-progress-reporting-to-BOF.txt ProgressableOutputHandler was supposed to handle that case, but it looks like it doesn't actually produce enough output to avoid the timeout, so instead this updated patch polls the future and reports the progress every couple of seconds until it completes. Add Progress Reporting to Cassandra OutputFormats - Key: CASSANDRA-3859 URL: https://issues.apache.org/jira/browse/CASSANDRA-3859 Project: Cassandra Issue Type: Improvement Components: Hadoop, Tools Affects Versions: 1.1 Reporter: Samarth Gahire Assignee: Brandon Williams Priority: Minor Labels: bulkloader, hadoop, mapreduce, sstableloader Fix For: 1.1 Attachments: 0001-add-progress-reporting-to-BOF.txt, 0002-Add-progress-to-CFOF.txt Original Estimate: 48h Remaining Estimate: 48h When we are using the BulkOutputFormat to load the data to cassandra. We should use the progress reporting to Hadoop Job within Sstable loader because while loading the data for particular task if streaming is taking more time and progress is not reported to Job it may kill the task with timeout exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3876) nodetool removetoken force causes an inconsistent state
[
https://issues.apache.org/jira/browse/CASSANDRA-3876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3876:
Reviewer: brandon.williams
Assignee: Sam Overton
nodetool removetoken force causes an inconsistent state
---
Key: CASSANDRA-3876
URL: https://issues.apache.org/jira/browse/CASSANDRA-3876
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.7, 1.1
Reporter: Sam Overton
Assignee: Sam Overton
Labels: force, nodetool, removetoken
Attachments: 3876.patch
Steps to reproduce (tested on 1.0.7 and trunk):
* Create a cluster of 3 nodes
* Insert some data
* stop one of the nodes
* Call removetoken on the token of the stopped node
* Immediately after, do removetoken force
- this will cause the original removetoken to fail with an error after 30s
since the generation changed for the leaving node, but this is a convenient
way of simulating the case where a removetoken hangs at streaming since the
cleanup logic at the end of StorageService.removeToken is never executed.
- if you want a more realistic reproduction then get a removetoken to hang
in streaming, then do removetoken force
Effects:
* removetoken status now throws an exception because
StorageService.removingNode is not cleared, but the endpoint is no longer a
member of the ring:
$ nodetool -h localhost removetoken status
{noformat}
Exception in thread main java.lang.AssertionError
at
org.apache.cassandra.locator.TokenMetadata.getToken(TokenMetadata.java:304)
at
org.apache.cassandra.service.StorageService.getRemovalStatus(StorageService.java:2369)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:111)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:45)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:226)
at
com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at
com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:205)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:683)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:672)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
at
javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:90)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1285)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1383)
at
javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:619)
at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
{noformat}
* truncate no longer works in the cli because the removed endpoint is not
removed from Gossiper.unreachableEndpoints.
The cli errors immediately with:
{noformat}
[default@ks1] truncate cf1;
null
UnavailableException()
at
org.apache.cassandra.thrift.Cassandra$truncate_result.read(Cassandra.java:20978)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.cassandra.thrift.Cassandra$Client.recv_truncate(Cassandra.java:942)
at
org.apache.cassandra.thrift.Cassandra$Client.truncate(Cassandra.java:929)
[jira] [Updated] (CASSANDRA-3371) Cassandra inferred schema and actual data don't match
[
https://issues.apache.org/jira/browse/CASSANDRA-3371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3371:
Attachment: 0002-Output-support-to-match-input.txt
0001-Rework-pig-schema.txt
New patches build upon v5 and adds output storage to match. The catch,
however, is that either the tuples (indexed/validated columns) or the bag
(unknown columns) are optional, so if you are dealing with narrow output, you
can just make a tuple like (key, (name,value), (name,value)) and that will work.
Cassandra inferred schema and actual data don't match
-
Key: CASSANDRA-3371
URL: https://issues.apache.org/jira/browse/CASSANDRA-3371
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 0.8.7
Reporter: Pete Warden
Assignee: Brandon Williams
Attachments: 0001-Rework-pig-schema.txt,
0002-Output-support-to-match-input.txt, 3371-v2.txt, 3371-v3.txt,
3371-v4.txt, 3371-v5-rebased.txt, 3371-v5.txt, pig.diff
It's looking like there may be a mismatch between the schema that's being
reported by the latest CassandraStorage.java, and the data that's actually
returned. Here's an example:
rows = LOAD 'cassandra://Frap/PhotoVotes' USING CassandraStorage();
DESCRIBE rows;
rows: {key: chararray,columns: {(name: chararray,value:
bytearray,photo_owner: chararray,value_photo_owner: bytearray,pid:
chararray,value_pid: bytearray,matched_string:
chararray,value_matched_string: bytearray,src_big: chararray,value_src_big:
bytearray,time: chararray,value_time: bytearray,vote_type:
chararray,value_vote_type: bytearray,voter: chararray,value_voter:
bytearray)}}
DUMP rows;
(691831038_1317937188.48955,{(photo_owner,1596090180),(pid,6855155124568798560),(matched_string,),(src_big,),(time,Thu
Oct 06 14:39:48 -0700 2011),(vote_type,album_dislike),(voter,691831038)})
getSchema() is reporting the columns as an inner bag of tuples, each of which
contains 16 values. In fact, getNext() seems to return an inner bag
containing 7 tuples, each of which contains two values.
It appears that things got out of sync with this change:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java?r1=1177083r2=1177082pathrev=1177083
See more discussion at:
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/pig-cassandra-problem-quot-Incompatible-field-schema-quot-error-tc6882703.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3677) NPE during HH delivery when gossip turned off on target
[ https://issues.apache.org/jira/browse/CASSANDRA-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3677: Fix Version/s: 1.0.8 NPE during HH delivery when gossip turned off on target --- Key: CASSANDRA-3677 URL: https://issues.apache.org/jira/browse/CASSANDRA-3677 Project: Cassandra Issue Type: Bug Affects Versions: 1.0.7 Reporter: Radim Kolar Assignee: Brandon Williams Priority: Trivial Fix For: 1.0.8 Attachments: 3677-v1.patch, 3677.txt probably not important bug ERROR [OptionalTasks:1] 2011-12-27 21:44:25,342 AbstractCassandraDaemon.java (line 138) Fatal exception in thread Thread[OptionalTasks:1,5,main] java.lang.NullPointerException at org.cliffc.high_scale_lib.NonBlockingHashMap.hash(NonBlockingHashMap.java:113) at org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:553) at org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:348) at org.cliffc.high_scale_lib.NonBlockingHashMap.putIfAbsent(NonBlockingHashMap.java:319) at org.cliffc.high_scale_lib.NonBlockingHashSet.add(NonBlockingHashSet.java:32) at org.apache.cassandra.db.HintedHandOffManager.scheduleHintDelivery(HintedHandOffManager.java:371) at org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:356) at org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:84) at org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:119) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3859) Add Progress Reporting to Cassandra SstableLoader for BulkOutputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3859: Attachment: 0002-Add-progress-to-CFOF.txt 0001-add-progress-reporting-to-BOF.txt First patch adds our own Progressable class so we can dodge most of the mapred/mapreduce mess and adds it to BOF in BRW where we report progress anytime the loader reports output. Second patch builds on this and adds similar support to CFOF, since it also doesn't report progress and times out on occasion. Add Progress Reporting to Cassandra SstableLoader for BulkOutputFormat -- Key: CASSANDRA-3859 URL: https://issues.apache.org/jira/browse/CASSANDRA-3859 Project: Cassandra Issue Type: Improvement Components: Hadoop, Tools Affects Versions: 1.1 Reporter: Samarth Gahire Assignee: Brandon Williams Priority: Minor Labels: bulkloader, hadoop, mapreduce, sstableloader Fix For: 1.1 Attachments: 0001-add-progress-reporting-to-BOF.txt, 0002-Add-progress-to-CFOF.txt Original Estimate: 48h Remaining Estimate: 48h When we are using the BulkOutputFormat to load the data to cassandra. We should use the progress reporting to Hadoop Job within Sstable loader because while loading the data for particular task if streaming is taking more time and progress is not reported to Job it may kill the task with timeout exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3859) Add Progress Reporting to Cassandra SstableLoader for BulkOutputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3859: Attachment: (was: 0001-add-progress-reporting-to-BOF.txt) Add Progress Reporting to Cassandra SstableLoader for BulkOutputFormat -- Key: CASSANDRA-3859 URL: https://issues.apache.org/jira/browse/CASSANDRA-3859 Project: Cassandra Issue Type: Improvement Components: Hadoop, Tools Affects Versions: 1.1 Reporter: Samarth Gahire Assignee: Brandon Williams Priority: Minor Labels: bulkloader, hadoop, mapreduce, sstableloader Fix For: 1.1 Attachments: 0001-add-progress-reporting-to-BOF.txt, 0002-Add-progress-to-CFOF.txt Original Estimate: 48h Remaining Estimate: 48h When we are using the BulkOutputFormat to load the data to cassandra. We should use the progress reporting to Hadoop Job within Sstable loader because while loading the data for particular task if streaming is taking more time and progress is not reported to Job it may kill the task with timeout exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3859) Add Progress Reporting to Cassandra SstableLoader for BulkOutputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3859: Attachment: (was: 0002-Add-progress-to-CFOF.txt) Add Progress Reporting to Cassandra SstableLoader for BulkOutputFormat -- Key: CASSANDRA-3859 URL: https://issues.apache.org/jira/browse/CASSANDRA-3859 Project: Cassandra Issue Type: Improvement Components: Hadoop, Tools Affects Versions: 1.1 Reporter: Samarth Gahire Assignee: Brandon Williams Priority: Minor Labels: bulkloader, hadoop, mapreduce, sstableloader Fix For: 1.1 Attachments: 0001-add-progress-reporting-to-BOF.txt, 0002-Add-progress-to-CFOF.txt Original Estimate: 48h Remaining Estimate: 48h When we are using the BulkOutputFormat to load the data to cassandra. We should use the progress reporting to Hadoop Job within Sstable loader because while loading the data for particular task if streaming is taking more time and progress is not reported to Job it may kill the task with timeout exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3859) Add Progress Reporting to Cassandra SstableLoader for BulkOutputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3859: Attachment: 0002-Add-progress-to-CFOF.txt 0001-add-progress-reporting-to-BOF.txt Add Progress Reporting to Cassandra SstableLoader for BulkOutputFormat -- Key: CASSANDRA-3859 URL: https://issues.apache.org/jira/browse/CASSANDRA-3859 Project: Cassandra Issue Type: Improvement Components: Hadoop, Tools Affects Versions: 1.1 Reporter: Samarth Gahire Assignee: Brandon Williams Priority: Minor Labels: bulkloader, hadoop, mapreduce, sstableloader Fix For: 1.1 Attachments: 0001-add-progress-reporting-to-BOF.txt, 0002-Add-progress-to-CFOF.txt Original Estimate: 48h Remaining Estimate: 48h When we are using the BulkOutputFormat to load the data to cassandra. We should use the progress reporting to Hadoop Job within Sstable loader because while loading the data for particular task if streaming is taking more time and progress is not reported to Job it may kill the task with timeout exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3859) Add Progress Reporting to Cassandra OutputFormats
[ https://issues.apache.org/jira/browse/CASSANDRA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3859: Summary: Add Progress Reporting to Cassandra OutputFormats (was: Add Progress Reporting to Cassandra SstableLoader for BulkOutputFormat) Add Progress Reporting to Cassandra OutputFormats - Key: CASSANDRA-3859 URL: https://issues.apache.org/jira/browse/CASSANDRA-3859 Project: Cassandra Issue Type: Improvement Components: Hadoop, Tools Affects Versions: 1.1 Reporter: Samarth Gahire Assignee: Brandon Williams Priority: Minor Labels: bulkloader, hadoop, mapreduce, sstableloader Fix For: 1.1 Attachments: 0001-add-progress-reporting-to-BOF.txt, 0002-Add-progress-to-CFOF.txt Original Estimate: 48h Remaining Estimate: 48h When we are using the BulkOutputFormat to load the data to cassandra. We should use the progress reporting to Hadoop Job within Sstable loader because while loading the data for particular task if streaming is taking more time and progress is not reported to Job it may kill the task with timeout exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3847) Pig should throw a useful error when the destination CF doesn't exist
[
https://issues.apache.org/jira/browse/CASSANDRA-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3847:
Attachment: 3847.txt
Unfortunately, there's nothing we can do about pig catching errors and
rethrowing the org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2042:
Error in new logical plan. Try -Dpig.usenewlogicalplan=false. message, but we
can at least throw a helpful error into the log instead of an NPE.
Pig should throw a useful error when the destination CF doesn't exist
-
Key: CASSANDRA-3847
URL: https://issues.apache.org/jira/browse/CASSANDRA-3847
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Brandon Williams
Fix For: 1.0.8
Attachments: 3847.txt
When trying to store data to nonexistent CF, no good error is returned.
Instead you get a message like:
{noformat}
[main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2042: Error in new
logical plan. Try -Dpig.usenewlogicalplan=false.
{noformat}
Which, if you follow its advice, will eventually lead you to an NPE in
initSchema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3847) Pig should throw a useful error when the destination CF doesn't exist
[
https://issues.apache.org/jira/browse/CASSANDRA-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3847:
Reviewer: xedin
Pig should throw a useful error when the destination CF doesn't exist
-
Key: CASSANDRA-3847
URL: https://issues.apache.org/jira/browse/CASSANDRA-3847
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Brandon Williams
Fix For: 1.0.8
Attachments: 3847.txt
When trying to store data to nonexistent CF, no good error is returned.
Instead you get a message like:
{noformat}
[main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2042: Error in new
logical plan. Try -Dpig.usenewlogicalplan=false.
{noformat}
Which, if you follow its advice, will eventually lead you to an NPE in
initSchema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
[
https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3740:
Attachment: 0003-use-output-partitioner.txt
Fix incorrect variable scope causing NPE.
While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
--
Key: CASSANDRA-3740
URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1
Reporter: Samarth Gahire
Assignee: Brandon Williams
Labels: cassandra, hadoop, mapreduce
Fix For: 1.1
Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt,
0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt,
0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
I am trying to use BulkOutputFormat to stream the data from map of Hadoop
job. I have set the cassandra related configuration using ConfigHelper ,Also
have looked into Cassandra code seems Cassandra has taken care that it should
not look for the cassandra.yaml file.
But still when I run the job i get the following error:
{
12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing
the arguments. Applications should implement Tool for the same.
12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
12/01/13 11:30:23 INFO mapred.JobClient: Task Id :
attempt_201201130910_0015_m_00_0, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
attempt_201201130910_0015_m_00_0: Cannot locate cassandra.yaml
attempt_201201130910_0015_m_00_0: Fatal configuration error; unable to
start server.
}
Also let me know how can i make this cassandra.yaml file available to Hadoop
mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
[
https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3740:
Attachment: (was: 0003-use-output-partitioner.txt)
While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
--
Key: CASSANDRA-3740
URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1
Reporter: Samarth Gahire
Assignee: Brandon Williams
Labels: cassandra, hadoop, mapreduce
Fix For: 1.1
Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt,
0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt,
0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
I am trying to use BulkOutputFormat to stream the data from map of Hadoop
job. I have set the cassandra related configuration using ConfigHelper ,Also
have looked into Cassandra code seems Cassandra has taken care that it should
not look for the cassandra.yaml file.
But still when I run the job i get the following error:
{
12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing
the arguments. Applications should implement Tool for the same.
12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
12/01/13 11:30:23 INFO mapred.JobClient: Task Id :
attempt_201201130910_0015_m_00_0, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
attempt_201201130910_0015_m_00_0: Cannot locate cassandra.yaml
attempt_201201130910_0015_m_00_0: Fatal configuration error; unable to
start server.
}
Also let me know how can i make this cassandra.yaml file available to Hadoop
mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3754) Add TTL support to BulkOutputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3754: Summary: Add TTL support to BulkOutputFormat (was: Add TTL and counter support to BulkOutputFormat) Add TTL support to BulkOutputFormat --- Key: CASSANDRA-3754 URL: https://issues.apache.org/jira/browse/CASSANDRA-3754 Project: Cassandra Issue Type: Improvement Components: Hadoop Affects Versions: 1.1 Reporter: Brandon Williams Assignee: Samarth Gahire Priority: Minor Labels: bulkloader, hadoop, ttl Fix For: 1.1 Attachments: 0001-TTL-Support-Added -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3840) Use java.io.tmpdir as default output location for BulkRecordWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3840: Affects Version/s: (was: 1.1) Assignee: Erik Forsberg Use java.io.tmpdir as default output location for BulkRecordWriter -- Key: CASSANDRA-3840 URL: https://issues.apache.org/jira/browse/CASSANDRA-3840 Project: Cassandra Issue Type: Improvement Components: Hadoop Reporter: Erik Forsberg Assignee: Erik Forsberg Labels: bulkloader Fix For: 1.1 Attachments: java.io.tmpdir.patch BulkRecordWriter uses the value of the property mapreduce.output.bulkoutputformat.localdir if set, defaulting to value of mapred.local.dir if the former is not set. However, on a typical production system, mapred.local.dir is set to a list of directories. This leads to BulkOutputFormat writing to silly paths such as /dir1/,dir2,/dir3,KeySpaceName/CFName This has two effects: 1) Directory is not removed when job is finished, leading to disk space management issues. 2) If a new job is run against same keyspacename and CF, it tries to load old data + new data. Better to use System.getProperty(java.io.tmpdir), as that is set to an attempt-specific temporary directory which is cleaned after the job finishes. See http://hadoop.apache.org/common/docs/current/mapred_tutorial.html, under Directory Structure. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3826) Pig cannot use output formats other than CFOF
[ https://issues.apache.org/jira/browse/CASSANDRA-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3826: Attachment: (was: 3826-trunk.txt) Pig cannot use output formats other than CFOF - Key: CASSANDRA-3826 URL: https://issues.apache.org/jira/browse/CASSANDRA-3826 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.0.8 Pig has ColumnFamilyOutputFormat hard coded. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3826) Pig cannot use output formats other than CFOF
[ https://issues.apache.org/jira/browse/CASSANDRA-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3826: Attachment: (was: 3826.txt) Pig cannot use output formats other than CFOF - Key: CASSANDRA-3826 URL: https://issues.apache.org/jira/browse/CASSANDRA-3826 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 1.0.8 Pig has ColumnFamilyOutputFormat hard coded. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
