[jira] [Created] (CASSANDRA-10854) cqlsh COPY FROM csv having line with more than one consecutive ',' delimiter is throwing 'list index out of range'
Puspendu Banerjee created CASSANDRA-10854: - Summary: cqlsh COPY FROM csv having line with more than one consecutive ',' delimiter is throwing 'list index out of range' Key: CASSANDRA-10854 URL: https://issues.apache.org/jira/browse/CASSANDRA-10854 Project: Cassandra Issue Type: Bug Components: Tools Environment: cqlsh 5.0.1 | Cassandra 2.1.11.969 | DSE 4.8.3 | CQL spec 3.2.1 Reporter: Puspendu Banerjee cqlsh COPY FROM csv having line with more than one consecutive ',' delimiter is throwing 'list index out of range' Steps to re-produce: CREATE TABLE tracks_by_album ( album_title TEXT, album_year INT, performer TEXT STATIC, album_genre TEXT STATIC, track_number INT, track_title TEXT, PRIMARY KEY ((album_title, album_year), track_number) ); Create a file: tracks_by_album.csv having following 2 lines : album,year,performer,genre,number,title a,2015,b c d,e f g,, cqlsh> COPY music.tracks_by_album (album_title, album_year, performer, album_genre, track_number, track_title) FROM '~/tracks_by_album.csv' WITH HEADER = 'true'; Error : Starting copy of music.tracks_by_album with columns ['album_title', 'album_year', 'performer', 'album_genre', 'track_number', 'track_title']. list index out of range Aborting import at record #1. Previously inserted records are still present, and some records after that may be present as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10850) v4 spec has tons of grammatical mistakes
[ https://issues.apache.org/jira/browse/CASSANDRA-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054169#comment-15054169 ] Sandeep Tamhankar commented on CASSANDRA-10850: --- Yeah, that's what I figured. Please do assign it to me and I'll try and take care of it sometime next week. > v4 spec has tons of grammatical mistakes > > > Key: CASSANDRA-10850 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10850 > Project: Cassandra > Issue Type: Bug > Components: Documentation and Website >Reporter: Sandeep Tamhankar > > https://github.com/apache/cassandra/blob/cassandra-3.0/doc/native_protocol_v4.spec > I notice the following in the first section of the spec and then gave up: > "The list of allowed opcode is defined Section 2.3" => "The list of allowed > opcode*s* is defined in Section 2.3" > "the details of each corresponding message is described Section 4" => "the > details of each corresponding message are described in Section 4" since the > subject is details, not message. > "Requests are those frame sent by" => "Requests are those frame*s* sent by" > I think someone should go through the whole spec and fix all the mistakes > rather than me pointing out the ones I notice piece-meal. I found the grammar > errors to be rather distracting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10852) Add requireAuthorization method to IAuthorizer
[ https://issues.apache.org/jira/browse/CASSANDRA-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054365#comment-15054365 ] Mike Adamson commented on CASSANDRA-10852: -- Branch and tests: ||trunk|| |[branch|https://github.com/mike-tr-adamson/cassandra/tree/10852-trunk]| |[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10852-trunk-testall/]| |[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10852-trunk-dtest/]| > Add requireAuthorization method to IAuthorizer > -- > > Key: CASSANDRA-10852 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10852 > Project: Cassandra > Issue Type: New Feature >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Minor > Fix For: 3.2 > > > The {{IAuthenticator}} interface has a {{requireAuthentication}} to indicate > whether an implementation requires an explicit login. For consistency it > would be useful for 3rd party implementers if the {{IAuthorizer}} had a > similar method {{requireAuthorization}} that would indicate if the > implementation required explicit authorization. > This would mean that we could remove and explicit {{instanceof}} checks for > {{AllowAllAuthenticator}} and {{AllowAllAuthorizer}} in the code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10580) On dropped mutations, more details should be logged.
[ https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054502#comment-15054502 ] Anubhav Kale commented on CASSANDRA-10580: -- Thanks for the interesting suggestion. Actually I considered going down that route when I started working on it, but I just wasn't sure what the rationale / design philosophy behind adding new metrics was therefore took a simpler route. Glad to see your feedback. I have attached 10580-metrics.patch and will open a separate JIRA for doing this on a CF basis. I am using ApproximateTime class wherever its not taking part in decision of dropping the mutation and simply used for logging. I hope that makes sense. I can clean up the methods in MessagingService a bit more if you like (couple of them are printing the same message). I wanted to send this out first to make sure I was on the right path. Also, a question: It appears that Timer.Update appends entries to the metric (which is what we want). Do you know at what point it starts dropping new appends / starts giving up ? I wonder if there is a huge number of dropped mutations will the timeTaken metric mess up ? To make this work for CF, I will probably pass the mutation to MessagingService.LogDroppedMessages (maybe through an overload) and update the metrics on appropriate CF. Does that make sense ? If this change looks good, I am more inclined towards making this work for CF before making up patches for old branches. Let me know if that's okay. Appreciate your time and feedback ! > On dropped mutations, more details should be logged. > > > Key: CASSANDRA-10580 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10580 > Project: Cassandra > Issue Type: Improvement > Components: Coordination > Environment: Production >Reporter: Anubhav Kale >Assignee: Anubhav Kale >Priority: Minor > Fix For: 3.2, 2.2.x > > Attachments: 10580-Metrics.patch, 10580.patch, > CASSANDRA-10580-Head.patch, Trunk.patch > > > In our production cluster, we are seeing a large number of dropped mutations. > At a minimum, we should print the time the thread took to get scheduled > thereby dropping the mutation (We should also print the Message / Mutation so > it helps in figuring out which column family was affected). This will help > find the right tuning parameter for write_timeout_in_ms. > The change is small and is in StorageProxy.java and MessagingTask.java. I > will submit a patch shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10853) deb package migration to dh_python2
Michael Shuler created CASSANDRA-10853: -- Summary: deb package migration to dh_python2 Key: CASSANDRA-10853 URL: https://issues.apache.org/jira/browse/CASSANDRA-10853 Project: Cassandra Issue Type: Task Components: Packaging Reporter: Michael Shuler Assignee: Michael Shuler Priority: Minor Fix For: 3.x I'm working on a deb job in jenkins, and I had forgotten to open a bug for this. There is no urgent need, since {{python-support}} is in Jessie, but this package is currently in transition to be removed. http://deb.li/dhs2p During deb build: {noformat} dh_pysupport: This program is deprecated, you should use dh_python2 instead. Migration guide: http://deb.li/dhs2p {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10580) On dropped mutations, more details should be logged.
[ https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Kale updated CASSANDRA-10580: - Attachment: 10580-Metrics.patch > On dropped mutations, more details should be logged. > > > Key: CASSANDRA-10580 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10580 > Project: Cassandra > Issue Type: Improvement > Components: Coordination > Environment: Production >Reporter: Anubhav Kale >Assignee: Anubhav Kale >Priority: Minor > Fix For: 3.2, 2.2.x > > Attachments: 10580-Metrics.patch, 10580.patch, > CASSANDRA-10580-Head.patch, Trunk.patch > > > In our production cluster, we are seeing a large number of dropped mutations. > At a minimum, we should print the time the thread took to get scheduled > thereby dropping the mutation (We should also print the Message / Mutation so > it helps in figuring out which column family was affected). This will help > find the right tuning parameter for write_timeout_in_ms. > The change is small and is in StorageProxy.java and MessagingTask.java. I > will submit a patch shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10850) v4 spec has tons of grammatical mistakes
[ https://issues.apache.org/jira/browse/CASSANDRA-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054334#comment-15054334 ] Robert Stupp commented on CASSANDRA-10850: -- Ok, go ahead :) > v4 spec has tons of grammatical mistakes > > > Key: CASSANDRA-10850 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10850 > Project: Cassandra > Issue Type: Bug > Components: Documentation and Website >Reporter: Sandeep Tamhankar > > https://github.com/apache/cassandra/blob/cassandra-3.0/doc/native_protocol_v4.spec > I notice the following in the first section of the spec and then gave up: > "The list of allowed opcode is defined Section 2.3" => "The list of allowed > opcode*s* is defined in Section 2.3" > "the details of each corresponding message is described Section 4" => "the > details of each corresponding message are described in Section 4" since the > subject is details, not message. > "Requests are those frame sent by" => "Requests are those frame*s* sent by" > I think someone should go through the whole spec and fix all the mistakes > rather than me pointing out the ones I notice piece-meal. I found the grammar > errors to be rather distracting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10852) Add requireAuthorization method to IAuthorizer
Mike Adamson created CASSANDRA-10852: Summary: Add requireAuthorization method to IAuthorizer Key: CASSANDRA-10852 URL: https://issues.apache.org/jira/browse/CASSANDRA-10852 Project: Cassandra Issue Type: New Feature Reporter: Mike Adamson Assignee: Mike Adamson Priority: Minor Fix For: 3.2 The {{IAuthenticator}} interface has a {{requireAuthentication}} to indicate whether an implementation requires an explicit login. For consistency it would be useful for 3rd party implementers if the {{IAuthorizer}} had a similar method {{requireAuthorization}} that would indicate if the implementation required explicit authorization. This would mean that we could remove and explicit {{instanceof}} checks for {{AllowAllAuthenticator}} and {{AllowAllAuthorizer}} in the code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10580) On dropped mutations, more details should be logged.
[ https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Kale updated CASSANDRA-10580: - Attachment: 10580-Metrics.patch > On dropped mutations, more details should be logged. > > > Key: CASSANDRA-10580 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10580 > Project: Cassandra > Issue Type: Improvement > Components: Coordination > Environment: Production >Reporter: Anubhav Kale >Assignee: Anubhav Kale >Priority: Minor > Fix For: 3.2, 2.2.x > > Attachments: 10580-Metrics.patch, 10580.patch, > CASSANDRA-10580-Head.patch, Trunk.patch > > > In our production cluster, we are seeing a large number of dropped mutations. > At a minimum, we should print the time the thread took to get scheduled > thereby dropping the mutation (We should also print the Message / Mutation so > it helps in figuring out which column family was affected). This will help > find the right tuning parameter for write_timeout_in_ms. > The change is small and is in StorageProxy.java and MessagingTask.java. I > will submit a patch shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9813) cqlsh column header can be incorrect when no rows are returned
[ https://issues.apache.org/jira/browse/CASSANDRA-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054571#comment-15054571 ] Ariel Weisberg commented on CASSANDRA-9813: --- To set up auto jobs with Cassci send you PGP public key to [~philipthompson] and he will send a key you can use to set up a web hook in github. I think it's generally a good idea to avoid creating merge commits because as far as I can tell any time you create a merge commit on a branch you make it impossible to rebase the branch without recreating the merge edits again manually. Maybe if you squash the merge into the original commit you can get rebase to work. Maybe someone knows a way to get git to handle it better. Here are the tests running out of my repo[ |2.2|[utests|http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-CASSANDRA-9813-2.2-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-CASSANDRA-9813-2.2-dtest/]| |3.x|[utests|http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-CASSANDRA-9813-3.0-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-CASSANDRA-9813-3.0-dtest/]| > cqlsh column header can be incorrect when no rows are returned > -- > > Key: CASSANDRA-9813 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9813 > Project: Cassandra > Issue Type: Bug >Reporter: Aleksey Yeschenko >Assignee: Adam Holmberg > Labels: cqlsh > Fix For: 2.1.x, 2.2.x, 3.x > > Attachments: 9813-2.1.txt, Test-for-9813.txt > > > Upon migration, we internally create a pair of surrogate clustering/regular > columns for compact static tables. These shouldn't be exposed to the user. > That is, for the table > {code} > CREATE TABLE bar (k int, c int, PRIMARY KEY (k)) WITH COMPACT STORAGE; > {code} > {{SELECT * FROM bar}} should not be returning this result set: > {code} > cqlsh:test> select * from bar; > c | column1 | k | value > ---+-+---+--- > (0 rows) > {code} > Should only contain the defined {{c}} and {{k}} columns. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10580) On dropped mutations, more details should be logged.
[ https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Kale updated CASSANDRA-10580: - Attachment: (was: 10580-Metrics.patch) > On dropped mutations, more details should be logged. > > > Key: CASSANDRA-10580 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10580 > Project: Cassandra > Issue Type: Improvement > Components: Coordination > Environment: Production >Reporter: Anubhav Kale >Assignee: Anubhav Kale >Priority: Minor > Fix For: 3.2, 2.2.x > > Attachments: 10580-Metrics.patch, 10580.patch, > CASSANDRA-10580-Head.patch, Trunk.patch > > > In our production cluster, we are seeing a large number of dropped mutations. > At a minimum, we should print the time the thread took to get scheduled > thereby dropping the mutation (We should also print the Message / Mutation so > it helps in figuring out which column family was affected). This will help > find the right tuning parameter for write_timeout_in_ms. > The change is small and is in StorageProxy.java and MessagingTask.java. I > will submit a patch shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator
[ https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054761#comment-15054761 ] Ariel Weisberg commented on CASSANDRA-9318: --- I got two cstar jobs to complete. [This job is set to allow 16 megabytes of transactions per coordinator, and disabled reads until they come back down to 12 megabytes.|http://cstar.datastax.com/graph?command=one_job=d1e720c8-a125-11e5-9051-0256e416528f=op_rate=1_write=1_aggregates=true=0=6664.35=0=11883.3] [This job is set to allow 64 megabytes of transactions per coordinator, and disabled reads until they came back down to 60 megabytes.|http://cstar.datastax.com/graph?command=one_job=26853362-a127-11e5-80c2-0256e416528f=op_rate=1_write=1_aggregates=true=0=322.85=0=12972.3] The job with 64 megabytes in flight kind of looks like it failed after 300 seconds. I didn't expect the threshold for things to fall apart to be quite that low, but generally speaking yeah more data in flight tends to cause bad things to happen. So why did the second one fall apart? First off mad props to whomever started collecting the GC logs. Lot's of continual full GC at the end. Sure enough the heap is only 1 gigabyte. Are we seriously running all our performance tests with a default heap of 1 gigabyte? I don't think it failed due to in flight requests (only had 32 megabytes in flight). I think it up OOMed due to other heap pressure. For this in-flight request backpressure to work I think we need to include the weight of memtables when making the decision. I am going to bump up the heap and try again to see if I can reduce the impact of other heap pressure to the point that we can start buffering more requests in flight. > Bound the number of in-flight requests at the coordinator > - > > Key: CASSANDRA-9318 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9318 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths, Streaming and Messaging >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 2.1.x, 2.2.x > > > It's possible to somewhat bound the amount of load accepted into the cluster > by bounding the number of in-flight requests and request bytes. > An implementation might do something like track the number of outstanding > bytes and requests and if it reaches a high watermark disable read on client > connections until it goes back below some low watermark. > Need to make sure that disabling read on the client connection won't > introduce other issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk
Merge branch 'cassandra-3.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/779817fe Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/779817fe Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/779817fe Branch: refs/heads/trunk Commit: 779817fe71c1d0a891b414df8194d3350089044a Parents: f7666d5 a8b9e53 Author: Dave BrosiusAuthored: Sat Dec 12 18:20:40 2015 -0500 Committer: Dave Brosius Committed: Sat Dec 12 18:20:40 2015 -0500 -- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/779817fe/src/java/org/apache/cassandra/service/StorageProxy.java --
cassandra git commit: fix minor parameterized logging typo
Repository: cassandra Updated Branches: refs/heads/cassandra-3.0 c0d47bed9 -> a8b9e533f fix minor parameterized logging typo Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8b9e533 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8b9e533 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8b9e533 Branch: refs/heads/cassandra-3.0 Commit: a8b9e533fd4c394111d506b94fbda28759a2604b Parents: c0d47be Author: Dave BrosiusAuthored: Sat Dec 12 18:16:27 2015 -0500 Committer: Dave Brosius Committed: Sat Dec 12 18:16:27 2015 -0500 -- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8b9e533/src/java/org/apache/cassandra/service/StorageProxy.java -- diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java b/src/java/org/apache/cassandra/service/StorageProxy.java index 397b8b9..f8ed61d 100644 --- a/src/java/org/apache/cassandra/service/StorageProxy.java +++ b/src/java/org/apache/cassandra/service/StorageProxy.java @@ -723,7 +723,7 @@ public class StorageProxy implements StorageProxyMBean //if there are no paired endpoints there are probably range movements going on, //so we write to the local batchlog to replay later if (pendingEndpoints.isEmpty()) -logger.warn("Received base materialized view mutation for key %s that does not belong " + +logger.warn("Received base materialized view mutation for key {} that does not belong " + "to this node. There is probably a range movement happening (move or decommission)," + "but this node hasn't updated its ring metadata yet. Adding mutation to " + "local batchlog to be replayed later.",
[1/2] cassandra git commit: fix minor parameterized logging typo
Repository: cassandra Updated Branches: refs/heads/trunk f7666d5f2 -> 779817fe7 fix minor parameterized logging typo Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8b9e533 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8b9e533 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8b9e533 Branch: refs/heads/trunk Commit: a8b9e533fd4c394111d506b94fbda28759a2604b Parents: c0d47be Author: Dave BrosiusAuthored: Sat Dec 12 18:16:27 2015 -0500 Committer: Dave Brosius Committed: Sat Dec 12 18:16:27 2015 -0500 -- src/java/org/apache/cassandra/service/StorageProxy.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8b9e533/src/java/org/apache/cassandra/service/StorageProxy.java -- diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java b/src/java/org/apache/cassandra/service/StorageProxy.java index 397b8b9..f8ed61d 100644 --- a/src/java/org/apache/cassandra/service/StorageProxy.java +++ b/src/java/org/apache/cassandra/service/StorageProxy.java @@ -723,7 +723,7 @@ public class StorageProxy implements StorageProxyMBean //if there are no paired endpoints there are probably range movements going on, //so we write to the local batchlog to replay later if (pendingEndpoints.isEmpty()) -logger.warn("Received base materialized view mutation for key %s that does not belong " + +logger.warn("Received base materialized view mutation for key {} that does not belong " + "to this node. There is probably a range movement happening (move or decommission)," + "but this node hasn't updated its ring metadata yet. Adding mutation to " + "local batchlog to be replayed later.",