[jira] [Updated] (CASSANDRA-15196) Refactor inheritance around CommitLogSegment management and allocation
[ https://issues.apache.org/jira/browse/CASSANDRA-15196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-15196: Complexity: Normal Change Category: Code Clarity Status: Open (was: Triage Needed) > Refactor inheritance around CommitLogSegment management and allocation > -- > > Key: CASSANDRA-15196 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15196 > Project: Cassandra > Issue Type: Improvement > Components: Local/Commit Log >Reporter: Joshua McKenzie >Assignee: Joshua McKenzie >Priority: Normal > > Back when I wrote CASSANDRA-8844 and CASSANDRA-12148, a few things about the > CommitLog stood out to me that I wasn't in love with. At the time I was much > more of the mind of "When in Rome..." regarding some of our structures, our > naming, and specifically regarding our penchant for inheritance vs. > composition. Turns out I have an itch here I need to scratch. > This patch refactors the CommitLog in a few key ways: > * Removes inheritence for CommitLogSegmentManagerX, instead has a single > CommitLogSegmentManager and a CommitLogSegmentAllocator interface > ** This interface is implemented by CommitLogSegmentAllocatorStandard and > CommitLogSegmentAllocatorCDC > * Renames a few variables and methods within the segment manager to make > their purpose and role more clear from names alone (hopefully): > ** allocatingFrom --> activeSegment > ** allocatingFrom() --> getActiveSegment() > ** advanceAllocatingFrom() – switchToNewSegment() > ** activeSegments --> unflushedSegments > ** getActiveSegments --> getUnflushedSegments() > ** awaitAvailableSegment(...) --> awaitSegmentAllocation(...) > ** isStillAllocating() --> hasRoom() > * Reorders some of the "allocation-and-compare-in-while-loop-decl) to more > idiomatic style > * Adds some comments to various under-documented methods, mostly related to > CDC and timing > As far as we are in the 4.0 testing, I don't expect this to hit pre 4.0. > Given it's a relatively minor refactor in well-tested code, doesn't change > functionality, and doesn't impact any of the publicly exposed APIs in the > CommitLog ecosystem, I'd be comfortable with it going in a minor release > after 4.0 is out. But I'm open to alternative viewpoints. > Have run unit tests locally w/out issue on both regular and cdc-targets; need > to get CI against the branch internally for full dtest run and I'll post when > it's done. Just wanted to get visibility to this out there in case anyone was > curious or thinking of working on the same thing. > [ Link to branch on > github|https://github.com/apache/cassandra/compare/trunk...josh-mckenzie:clean_cl_comp] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15196) Refactor inheritance around CommitLogSegment management and allocation
Joshua McKenzie created CASSANDRA-15196: --- Summary: Refactor inheritance around CommitLogSegment management and allocation Key: CASSANDRA-15196 URL: https://issues.apache.org/jira/browse/CASSANDRA-15196 Project: Cassandra Issue Type: Improvement Components: Local/Commit Log Reporter: Joshua McKenzie Assignee: Joshua McKenzie Back when I wrote CASSANDRA-8844 and CASSANDRA-12148, a few things about the CommitLog stood out to me that I wasn't in love with. At the time I was much more of the mind of "When in Rome..." regarding some of our structures, our naming, and specifically regarding our penchant for inheritance vs. composition. Turns out I have an itch here I need to scratch. This patch refactors the CommitLog in a few key ways: * Removes inheritence for CommitLogSegmentManagerX, instead has a single CommitLogSegmentManager and a CommitLogSegmentAllocator interface ** This interface is implemented by CommitLogSegmentAllocatorStandard and CommitLogSegmentAllocatorCDC * Renames a few variables and methods within the segment manager to make their purpose and role more clear from names alone (hopefully): ** allocatingFrom --> activeSegment ** allocatingFrom() --> getActiveSegment() ** advanceAllocatingFrom() – switchToNewSegment() ** activeSegments --> unflushedSegments ** getActiveSegments --> getUnflushedSegments() ** awaitAvailableSegment(...) --> awaitSegmentAllocation(...) ** isStillAllocating() --> hasRoom() * Reorders some of the "allocation-and-compare-in-while-loop-decl) to more idiomatic style * Adds some comments to various under-documented methods, mostly related to CDC and timing As far as we are in the 4.0 testing, I don't expect this to hit pre 4.0. Given it's a relatively minor refactor in well-tested code, doesn't change functionality, and doesn't impact any of the publicly exposed APIs in the CommitLog ecosystem, I'd be comfortable with it going in a minor release after 4.0 is out. But I'm open to alternative viewpoints. Have run unit tests locally w/out issue on both regular and cdc-targets; need to get CI against the branch internally for full dtest run and I'll post when it's done. Just wanted to get visibility to this out there in case anyone was curious or thinking of working on the same thing. [ Link to branch on github|https://github.com/apache/cassandra/compare/trunk...josh-mckenzie:clean_cl_comp] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15161) Cassandra is not Schema-free?!
[ https://issues.apache.org/jira/browse/CASSANDRA-15161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-15161: Resolution: Invalid Status: Resolved (was: Triage Needed) > Cassandra is not Schema-free?! > -- > > Key: CASSANDRA-15161 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15161 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: arganzheng >Priority: Normal > > Cassandra claims to be a schema-free NoSQL, however I can not insert record > without define every column before I can insert it. > Someone please tell me I was wrong, or Cassandra is actually NOT schema-less. > Thanks in advance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15161) Cassandra is not Schema-free?!
[ https://issues.apache.org/jira/browse/CASSANDRA-15161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16864020#comment-16864020 ] Joshua McKenzie commented on CASSANDRA-15161: - Please take questions like this to the user mailing list. This Jira is for tracking development of the project. > Cassandra is not Schema-free?! > -- > > Key: CASSANDRA-15161 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15161 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: arganzheng >Priority: Normal > > Cassandra claims to be a schema-free NoSQL, however I can not insert record > without define every column before I can insert it. > Someone please tell me I was wrong, or Cassandra is actually NOT schema-less. > Thanks in advance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15137) Switch http to https URLs in build.xml
[ https://issues.apache.org/jira/browse/CASSANDRA-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-15137: Status: Ready to Commit (was: Review In Progress) > Switch http to https URLs in build.xml > -- > > Key: CASSANDRA-15137 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15137 > Project: Cassandra > Issue Type: Task > Components: Build, Dependencies >Reporter: Michael Shuler >Assignee: Michael Shuler >Priority: Normal > Fix For: 4.0 > > Attachments: 0001-Switch-http-to-https-URLs-in-build.xml.patch > > > Switch to using https URLs wherever possible in build.xml. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15137) Switch http to https URLs in build.xml
[ https://issues.apache.org/jira/browse/CASSANDRA-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-15137: Status: Review In Progress (was: Patch Available) > Switch http to https URLs in build.xml > -- > > Key: CASSANDRA-15137 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15137 > Project: Cassandra > Issue Type: Task > Components: Build, Dependencies >Reporter: Michael Shuler >Assignee: Michael Shuler >Priority: Normal > Fix For: 4.0 > > Attachments: 0001-Switch-http-to-https-URLs-in-build.xml.patch > > > Switch to using https URLs wherever possible in build.xml. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15137) Switch http to https URLs in build.xml
[ https://issues.apache.org/jira/browse/CASSANDRA-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-15137: Reviewers: Joshua McKenzie > Switch http to https URLs in build.xml > -- > > Key: CASSANDRA-15137 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15137 > Project: Cassandra > Issue Type: Task > Components: Build, Dependencies >Reporter: Michael Shuler >Assignee: Michael Shuler >Priority: Normal > Fix For: 4.0 > > Attachments: 0001-Switch-http-to-https-URLs-in-build.xml.patch > > > Switch to using https URLs wherever possible in build.xml. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15137) Switch http to https URLs in build.xml
[ https://issues.apache.org/jira/browse/CASSANDRA-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844803#comment-16844803 ] Joshua McKenzie commented on CASSANDRA-15137: - Haven't tested all the paths, but assuming you have (and it's anyway on trunk as we have runway to fix if there's edge-cases this breaks), +1. > Switch http to https URLs in build.xml > -- > > Key: CASSANDRA-15137 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15137 > Project: Cassandra > Issue Type: Task > Components: Build, Dependencies >Reporter: Michael Shuler >Assignee: Michael Shuler >Priority: Normal > Fix For: 4.0 > > Attachments: 0001-Switch-http-to-https-URLs-in-build.xml.patch > > > Switch to using https URLs wherever possible in build.xml. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15137) Switch http to https URLs in build.xml
[ https://issues.apache.org/jira/browse/CASSANDRA-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844803#comment-16844803 ] Joshua McKenzie edited comment on CASSANDRA-15137 at 5/21/19 12:59 PM: --- Haven't tested all the paths, but assuming you have (and it's on trunk anyway so we have runway to fix if there's edge-cases this breaks), +1. was (Author: joshuamckenzie): Haven't tested all the paths, but assuming you have (and it's anyway on trunk as we have runway to fix if there's edge-cases this breaks), +1. > Switch http to https URLs in build.xml > -- > > Key: CASSANDRA-15137 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15137 > Project: Cassandra > Issue Type: Task > Components: Build, Dependencies >Reporter: Michael Shuler >Assignee: Michael Shuler >Priority: Normal > Fix For: 4.0 > > Attachments: 0001-Switch-http-to-https-URLs-in-build.xml.patch > > > Switch to using https URLs wherever possible in build.xml. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15106) Fat Cassandra and CQL nearer to SQL
[ https://issues.apache.org/jira/browse/CASSANDRA-15106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831110#comment-16831110 ] Joshua McKenzie commented on CASSANDRA-15106: - Thanks for that [~rustyrazorblade]; I totally should have thought to include that - sorry about that oversight Piotr! > Fat Cassandra and CQL nearer to SQL > --- > > Key: CASSANDRA-15106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15106 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/CQL >Reporter: Piotr Pastuszka >Priority: Normal > > I understand that Cassandra is a dispersed lightweight, performance / > scalability / reliability database. For this reason, it has limitations not > to overload the single node. > But these restrictions for someone who administers the database or explains > the problems are too big. They remind us of the attempt to use mono monitors > in the era of color monitors. > At the beginning I worked on Cassandra 2.0. At that time it was not possible > to use IN,>, <. Also, searching for several partitions was not possible. > Now in 3.11.4 it's all there is, but there are several necessary > functionalities that are very missing. > At the moment, to make the functions available, you need to change the > parameter in the configuration. > You could add below features and parameters to enable: > a) sorting on Cassandra's side (this could be done by the coordinator): .. > order by xxx. Sorting mechanisms are available in Cassandra because data is > sorted in partitions and there are indexes > b) subqueries. CASSANDRA-8846 SubQuery and IN operator implementation > c) CTAS - CREATE TABLE AS SELECT - CASSANDRA-8234 and INSERT INTO .. SELECT > ... > d) DBLINK to other databases (Oracle, MSsql) > e) functions in which the CQL query can be called and return a result > f) external table (file on disk visible as a table under database). > > I know there is SPARK, but it's a programming solution, not for > administrators or testers. > I am the administrator of Oracle and Cassandra. > I know SQL quite well and working with CQL is a torment. > The CQL editor is _very_ good, but CQL restrictions make it difficult to > solve any problem. > I can create all points a) -f) as independent request, but as I can see, all > such ideas are rejected / postponed. > The point is, however, to change the approach. Today's huge Cassandra's > restrictions reject many potential users. > Maybe you can combine Cassandara and Spark and give additional functionality > to CQL, or do it only on Cassandra side. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15106) Fat Cassandra and CQL nearer to SQL
[ https://issues.apache.org/jira/browse/CASSANDRA-15106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-15106: Resolution: Invalid Status: Resolved (was: Triage Needed) While you have a lot of ideas worth engaging with and discussing, the user mailing list or perhaps dev mailing list are a more appropriate place for discussions like this. The Jira project is specifically for active work on execution or more concrete debate about implementation; for higher level "what ifs" like this, the mailing lists are a better place to get the engagement you're looking for. > Fat Cassandra and CQL nearer to SQL > --- > > Key: CASSANDRA-15106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15106 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/CQL >Reporter: Piotr Pastuszka >Priority: Normal > > I understand that Cassandra is a dispersed lightweight, performance / > scalability / reliability database. For this reason, it has limitations not > to overload the single node. > But these restrictions for someone who administers the database or explains > the problems are too big. They remind us of the attempt to use mono monitors > in the era of color monitors. > At the beginning I worked on Cassandra 2.0. At that time it was not possible > to use IN,>, <. Also, searching for several partitions was not possible. > Now in 3.11.4 it's all there is, but there are several necessary > functionalities that are very missing. > At the moment, to make the functions available, you need to change the > parameter in the configuration. > You could add below features and parameters to enable: > a) sorting on Cassandra's side (this could be done by the coordinator): .. > order by xxx. Sorting mechanisms are available in Cassandra because data is > sorted in partitions and there are indexes > b) subqueries. CASSANDRA-8846 SubQuery and IN operator implementation > c) CTAS - CREATE TABLE AS SELECT - CASSANDRA-8234 and INSERT INTO .. SELECT > ... > d) DBLINK to other databases (Oracle, MSsql) > e) functions in which the CQL query can be called and return a result > f) external table (file on disk visible as a table under database). > > I know there is SPARK, but it's a programming solution, not for > administrators or testers. > I am the administrator of Oracle and Cassandra. > I know SQL quite well and working with CQL is a torment. > The CQL editor is _very_ good, but CQL restrictions make it difficult to > solve any problem. > I can create all points a) -f) as independent request, but as I can see, all > such ideas are rejected / postponed. > The point is, however, to change the approach. Today's huge Cassandra's > restrictions reject many potential users. > Maybe you can combine Cassandara and Spark and give additional functionality > to CQL, or do it only on Cassandra side. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-8282) org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out while selecting query
[
https://issues.apache.org/jira/browse/CASSANDRA-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-8282:
---
Status: Resolved (was: Open)
> org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out
> while selecting query
> ---
>
> Key: CASSANDRA-8282
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8282
> Project: Cassandra
> Issue Type: Bug
> Environment: Ubuntu 12
> AWS network
> [cqlsh 5.0.1 | Cassandra 2.1.0 | CQL spec 3.2.0 | Native protocol v3]
> 4 cores and 15GB ram with 6 nodes.
>Reporter: murali
>Priority: Low
> Labels: performance
>
> Hi there,
> We are getting below error when we ran {{select count(*) from demo.songs}} on
> CQLSH ( [cqlsh 5.0.1 | Cassandra 2.1.0 | CQL spec 3.2.0 | Native protocol v3]
> ) and we are on AWS m3.large instances with 6 nodes in single DC.
> {noformat}
> ERROR [Thrift:15] 2014-11-10 09:27:52,613 CustomTThreadPoolServer.java:219 -
> Error occurred during processing of message.
> com.google.common.util.concurrent.UncheckedExecutionException:
> java.lang.RuntimeException:
> org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
> received only 0 responses.
> at
> com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
> ~[guava-16.0.jar:na]
> at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
> ~[guava-16.0.jar:na]
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
> ~[guava-16.0.jar:na]
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
> ~[guava-16.0.jar:na]
> at
> org.apache.cassandra.service.ClientState.authorize(ClientState.java:353)
> ~[apache-cassandra-2.1.0.jar:2.1.0]
> at
> org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:225)
> ~[apache-cassandra-2.1.0.jar:2.1.0]
> at
> org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:219)
> ~[apache-cassandra-2.1.0.jar:2.1.0]
> at
> org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:203)
> ~[apache-cassandra-2.1.0.jar:2.1.0]
> at
> org.apache.cassandra.thrift.CassandraServer.multiget_slice(CassandraServer.java:370)
> ~[apache-cassandra-2.1.0.jar:2.1.0]
> at
> org.apache.cassandra.thrift.Cassandra$Processor$multiget_slice.getResult(Cassandra.java:3716)
> ~[apache-cassandra-thrift-2.1.0.jar:2.1.0]
> at
> org.apache.cassandra.thrift.Cassandra$Processor$multiget_slice.getResult(Cassandra.java:3700)
> ~[apache-cassandra-thrift-2.1.0.jar:2.1.0]
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> ~[libthrift-0.9.1.jar:0.9.1]
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> ~[libthrift-0.9.1.jar:0.9.1]
> at
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:201)
> ~[apache-cassandra-2.1.0.jar:2.1.0]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> Caused by: java.lang.RuntimeException:
> org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
> received only 0 responses.
> at org.apache.cassandra.auth.Auth.selectUser(Auth.java:271)
> ~[apache-cassandra-2.1.0.jar:2.1.0]
> at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:88)
> ~[apache-cassandra-2.1.0.jar:2.1.0]
> at
> org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
> ~[apache-cassandra-2.1.0.jar:2.1.0]
> at
> org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:67)
> ~[apache-cassandra-2.1.0.jar:2.1.0]
> at
> org.apache.cassandra.service.ClientState$1.load(ClientState.java:339)
> ~[apache-cassandra-2.1.0.jar:2.1.0]
> at
> org.apache.cassandra.service.ClientState$1.load(ClientState.java:336)
> ~[apache-cassandra-2.1.0.jar:2.1.0]
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
> ~[guava-16.0.jar:na]
> at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
> ~[guava-16.0.jar:na]
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
> ~[guava-16.0.jar:na]
> at
> com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
> ~[guava-16.0.jar:na]
> ... 16 common frames omitted
> Caused by:
[jira] [Updated] (CASSANDRA-13433) RPM distribution improvements and known issues
[ https://issues.apache.org/jira/browse/CASSANDRA-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-13433: Status: Open (was: Testing) > RPM distribution improvements and known issues > -- > > Key: CASSANDRA-13433 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13433 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: Stefan Podkowinski >Assignee: Stefan Podkowinski >Priority: Major > Attachments: cassandra-3.9-centos6.patch > > > Starting with CASSANDRA-13252, new releases will be provided as both official > RPM and Debian packages. While the Debian packages are already well > established with our user base, the RPMs just have been release for the first > time and still require some attention. > Feel free to discuss RPM related issues in this ticket and open a sub-task to > fill a bug report. > Please note that native systemd support will be implemented with > CASSANDRA-13148 and this is not strictly a RPM specific issue. We still > intent to offer non-systemd support based on the already working init scripts > that we ship. Therefor the first step is to make use of systemd backward > compatibility for SysV/LSB scripts, so we can provide RPMs for both systemd > and non-systemd environments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13433) RPM distribution improvements and known issues
[ https://issues.apache.org/jira/browse/CASSANDRA-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-13433: Status: Awaiting Feedback (was: Open) > RPM distribution improvements and known issues > -- > > Key: CASSANDRA-13433 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13433 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: Stefan Podkowinski >Assignee: Stefan Podkowinski >Priority: Major > Attachments: cassandra-3.9-centos6.patch > > > Starting with CASSANDRA-13252, new releases will be provided as both official > RPM and Debian packages. While the Debian packages are already well > established with our user base, the RPMs just have been release for the first > time and still require some attention. > Feel free to discuss RPM related issues in this ticket and open a sub-task to > fill a bug report. > Please note that native systemd support will be implemented with > CASSANDRA-13148 and this is not strictly a RPM specific issue. We still > intent to offer non-systemd support based on the already working init scripts > that we ship. Therefor the first step is to make use of systemd backward > compatibility for SysV/LSB scripts, so we can provide RPMs for both systemd > and non-systemd environments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15015) Cassandra metrics documentation is not correct for Hint_delays metric
[ https://issues.apache.org/jira/browse/CASSANDRA-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-15015: Status: Open (was: Ready to Commit) > Cassandra metrics documentation is not correct for Hint_delays metric > - > > Key: CASSANDRA-15015 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15015 > Project: Cassandra > Issue Type: Bug > Components: Documentation/Blog >Reporter: Anup Shirolkar >Assignee: Anup Shirolkar >Priority: Minor > Labels: documentation, easyfix, low-hanging-fruit > Fix For: 3.0.19, 3.11.5, 4.0 > > Attachments: 150150-trunk.txt > > > The Cassandra metrics for hint delays are not correctly referred on the > documentation web page: > [http://cassandra.apache.org/doc/latest/operating/metrics.html#hintsservice-metrics] > > The metrics are defined in the > [code|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/metrics/HintsServiceMetrics.java#L45-L52] > as 'Hint_delays' and 'Hint_delays-' but those are listed on the > website as 'Hints_delays' and 'Hints_delays-'. > The documentation should be fixed by removing the extra 's' in Hints to match > it with code. > The Jira for adding hint_delays: CASSANDRA-13234 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15015) Cassandra metrics documentation is not correct for Hint_delays metric
[ https://issues.apache.org/jira/browse/CASSANDRA-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-15015: Status: Patch Available (was: Open) > Cassandra metrics documentation is not correct for Hint_delays metric > - > > Key: CASSANDRA-15015 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15015 > Project: Cassandra > Issue Type: Bug > Components: Documentation/Blog >Reporter: Anup Shirolkar >Assignee: Anup Shirolkar >Priority: Minor > Labels: documentation, easyfix, low-hanging-fruit > Fix For: 3.0.19, 3.11.5, 4.0 > > Attachments: 150150-trunk.txt > > > The Cassandra metrics for hint delays are not correctly referred on the > documentation web page: > [http://cassandra.apache.org/doc/latest/operating/metrics.html#hintsservice-metrics] > > The metrics are defined in the > [code|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/metrics/HintsServiceMetrics.java#L45-L52] > as 'Hint_delays' and 'Hint_delays-' but those are listed on the > website as 'Hints_delays' and 'Hints_delays-'. > The documentation should be fixed by removing the extra 's' in Hints to match > it with code. > The Jira for adding hint_delays: CASSANDRA-13234 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-10273) Reduce number of data directory scans during startup
[
https://issues.apache.org/jira/browse/CASSANDRA-10273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie reassigned CASSANDRA-10273:
---
Assignee: Shaurya Gupta (was: Giampaolo)
> Reduce number of data directory scans during startup
>
>
> Key: CASSANDRA-10273
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10273
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Startup and Shutdown
>Reporter: Robert Stupp
>Assignee: Shaurya Gupta
>Priority: Minor
> Labels: lhf
> Attachments: patch_CASSANDRA-10273_trunk
>
>
> ATM we scan each data directory four times. We could easily reduce that to at
> least two, maybe to one.
> # pre-flight (startup tests) scrub
> # pre-flight (startup tests) sstable min version
> # {{ColumnFamilyStore.createColumnFamilyStore}}
> # {{ColumnFamilyStore.}} (if {{loadSSTables==true}})
> First two pre-flight tests could be combined to one and 3+4 could also be
> combined, as both appear at pretty related code paths.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-11418) Nodetool status should reflect hibernate/replacing states
[ https://issues.apache.org/jira/browse/CASSANDRA-11418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie reassigned CASSANDRA-11418: --- Assignee: Shaurya Gupta > Nodetool status should reflect hibernate/replacing states > - > > Key: CASSANDRA-11418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11418 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Observability, Tool/nodetool >Reporter: Joel Knighton >Assignee: Shaurya Gupta >Priority: Minor > Labels: lhf > Fix For: 4.x > > Attachments: cassandra-11418-trunk > > > Currently, the four options for state in nodetool status are > joining/leaving/moving/normal. > Joining nodes are determined based on bootstrap tokens, leaving nodes are > based on leaving endpoints in TokenMetadata, moving nodes are based on moving > endpoints in TokenMetadata. > This means that a node will appear in normal state when going through a > bootstrap with flag replace_address, which can be confusing to operators. > We should add another state for hibernation/replacing to make this visible. > This will require a way to get a list of all hibernating endpoints. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14697) Transient Replication 4.0 pre-release followup work
[ https://issues.apache.org/jira/browse/CASSANDRA-14697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662667#comment-16662667 ] Joshua McKenzie edited comment on CASSANDRA-14697 at 10/24/18 6:48 PM: --- So just to throw this out there - TR was committed pre feature-freeze date, but having 23 linked follow-up items, many of them pretty core functionality to the feature, really makes it seem like this was merged in in a very incomplete state just to hit the freeze date. That concerns me. was (Author: joshuamckenzie): So just to throw this out there - TR was committed pre feature-freeze date, but having 23 linked follow-up items, many of them pretty core functionality to the feature, really makes it seem like this was merged in when it was just to hit the freeze date. That concerns me. > Transient Replication 4.0 pre-release followup work > --- > > Key: CASSANDRA-14697 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14697 > Project: Cassandra > Issue Type: Task >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Major > Fix For: 4.0 > > > This is an umbrella ticket for linking work done post CASSANDRA-14404. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14697) Transient Replication 4.0 pre-release followup work
[ https://issues.apache.org/jira/browse/CASSANDRA-14697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662667#comment-16662667 ] Joshua McKenzie commented on CASSANDRA-14697: - So just to throw this out there - TR was committed pre feature-freeze date, but having 23 linked follow-up items, many of them pretty core functionality to the feature, really makes it seem like this was merged in when it was just to hit the freeze date. That concerns me. > Transient Replication 4.0 pre-release followup work > --- > > Key: CASSANDRA-14697 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14697 > Project: Cassandra > Issue Type: Task >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Major > Fix For: 4.0 > > > This is an umbrella ticket for linking work done post CASSANDRA-14404. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14814) Can anyone give me some pointers where to look, or start to look for where this issue is coming from? On startup I get this in the logs and my service goes into a
[ https://issues.apache.org/jira/browse/CASSANDRA-14814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie resolved CASSANDRA-14814. - Resolution: Invalid [~Hacky44]: As indicated on the other ticket, you'll get faster, higher quality responses by reaching out on #cassandra on freenode or hitting up the user mailing lists. Closing as invalid. > Can anyone give me some pointers where to look, or start to look for where > this issue is coming from? On startup I get this in the logs and my service > goes into a failed state. Active(exited) > - > > Key: CASSANDRA-14814 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14814 > Project: Cassandra > Issue Type: Bug > Components: Configuration >Reporter: Ravi Gangwar >Priority: Major > Fix For: 2.2.x > > > Hi, > Can anyone give me some pointers where to look, or start to look for where > this issue is coming from? > On startup I get this in the logs and my service goes into a failed state. > Active(exited) > ERROR [main] 2018-08-28 16:25:01,443 CassandraDaemon.java:655 - Exception > encountered during startup > java.lang.IllegalStateException: Failed to bind port 9042 on 192.168.118.237. > at org.apache.cassandra.transport.Server.run(Server.java:187) > ~[apache-cassandra-2.2.11.jar:2.2.11] > at org.apache.cassandra.transport.Server.start(Server.java:120) > ~[apache-cassandra-2.2.11.jar:2.2.11] > at > org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:446) > [apache-cassandra-2.2.11.jar:2.2.11] > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:548) > [apache-cassandra-2.2.11.jar:2.2.11] > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:642) > [apache-cassandra-2.2.11.jar:2.2.11] > WARN [main] 2018-08-28 16:24:57,894 CLibrary.java:159 - Unable to lock JVM > memory (ENOMEM). This can result in part of the JVM being swapped out, > especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra > as root. > WARN [main] 2018-08-28 16:24:57,894 StartupChecks.java:118 - jemalloc shared > library could not be preloaded to speed up memory allocations > WARN [main] 2018-08-28 16:24:57,895 StartupChecks.java:150 - JMX is not > enabled to receive remote connections. Please see cassandra-env.sh for more > info. > WARN [SharedPool-Worker-6] 2018-08-28 16:28:21,624 SliceQueryFilter.java:307 > - Read 643 live and 1286 tombstone cells in system.schema_columns for key: > hdb (see tombstone_warn_threshold). 2147483647 columns were requested, > slices=[-] > WARN [SharedPool-Worker-4] 2018-08-28 16:29:22,318 SliceQueryFilter.java:307 > - Read 643 live and 1286 tombstone cells in system.schema_columns for key: > hdb (see tombstone_warn_threshold). 2147483647 columns were requested, > slices=[-] > WARN [main] 2018-08-28 16:55:09,948 StartupChecks.java:118 - jemalloc shared > library could not be preloaded to speed up memory allocations > WARN [main] 2018-08-28 16:55:09,948 StartupChecks.java:150 - JMX is not > enabled to receive remote connections. Please see cassandra-env.sh for more > info. > > Regards, > Ravi -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14805) Fails on running Cassandra server
[ https://issues.apache.org/jira/browse/CASSANDRA-14805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie resolved CASSANDRA-14805. - Resolution: Invalid > Fails on running Cassandra server > -- > > Key: CASSANDRA-14805 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14805 > Project: Cassandra > Issue Type: Bug >Reporter: Ravi Gangwar >Priority: Critical > > Full Product Version : > Fails on java version. It's Java "1.8.0_181" > Os Version : Ubuntu 16.04 LTS > EXTRA RELEVANT SYSTEM CONFIGURATION : > Just installed Cassandra 2.2.11 > A DESCRIPTION OF THE PROBLEM : > # > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGBUS (0x7) at pc=0x7f42fc492e70, pid=12128, tid=0x7f42fc3c9700 > # > # JRE version: Java(TM) SE Runtime Environment (8.0_181-b13) (build > 1.8.0_181-b13) > # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.181-b13 mixed mode > linux-amd64 compressed oops) > # Problematic frame: > # C [liblz4-java8098863625230398555.so+0x5e70] LZ4_decompress_fast+0xd0 > # > # Failed to write core dump. Core dumps have been disabled. To enable core > dumping, try "ulimit -c unlimited" before starting Java again > # > # If you would like to submit a bug report, please visit: > # http://bugreport.java.com/bugreport/crash.jsp > # The crash happened outside the Java Virtual Machine in native code. > # See problematic frame for where to report the bug. > # > --- T H R E A D --- > Current thread (0x006d0800): JavaThread "CompactionExecutor:7" daemon > [_thread_in_native, id=5278, stack(0x7f42fc389000,0x7f42fc3ca000)] > siginfo: si_signo: 7 (SIGBUS), si_code: 2 (BUS_ADRERR), si_addr: > 0x7f2026b8b000 > Registers: > RAX=0x7f432200750a, RBX=0x7f2026b8affc, RCX=0x7f4322005856, > RDX=0x7f432200750a > RSP=0x7f42fc3c7e50, RBP=0x7f2026b8940f, RSI=0x7f4322011d88, > RDI=0x0007 > R8 =0x7f4322011d84, R9 =0x7f4322011d90, R10=0x0003, > R11=0x > R12=0x, R13=0x7f1ffe33d40f, R14=0x7f4322011d87, > R15=0x7f4322011d8b > RIP=0x7f42fc492e70, EFLAGS=0x00010287, CSGSFS=0x002b0033, > ERR=0x0004 > TRAPNO=0x000e > Top of Stack: (sp=0x7f42fc3c7e50) > 0x7f42fc3c7e50: 4ce9 > 0x7f42fc3c7e60: 0004 0001 > 0x7f42fc3c7e70: 0002 0001 > 0x7f42fc3c7e80: 0004 0004 > 0x7f42fc3c7e90: 0004 0004 > 0x7f42fc3c7ea0: > 0x7f42fc3c7eb0: > 0x7f42fc3c7ec0: 0001 > 0x7f42fc3c7ed0: 0002 0003 > 0x7f42fc3c7ee0: 7f42fc3c7fa8 006d09f8 > 0x7f42fc3c7ef0: > 0x7f42fc3c7f00: 7f1ffe33d40f 7f4322001d90 > 0x7f42fc3c7f10: 2884c000 7f42fc48f59d > 0x7f42fc3c7f20: 000733b5fa78 > 0x7f42fc3c7f30: 7f42fc3c7fc0 > 0x7f42fc3c7f40: 2884bfff 7f42fc3c7fa8 > 0x7f42fc3c7f50: 006d0800 7f43119e8f1d > 0x7f42fc3c7f60: 7f42fc3c7f98 > 0x7f42fc3c7f70: 0001 > 0x7f42fc3c7f80: 00070764f4d8 > 0x7f42fc3c7f90: 00075b5937f0 > 0x7f42fc3c7fa0: 7f42fc3c7ff0 0006f91d2e38 > 0x7f42fc3c7fb0: 0006d7e3f210 7f42fc3c8040 > 0x7f42fc3c7fc0: 00070764f4d8 7f4311f9eca8 > 0x7f42fc3c7fd0: 51070001 00072dc9abb0 > 0x7f42fc3c7fe0: 28851103 0006dd2d > 0x7f42fc3c7ff0: 00022883f40b eb6b26fe2884bffc > 0x7f42fc3c8000: 00070764f4d8 00012884c000 > 0x7f42fc3c8010: 00075b5937f0 00075b5937f0 > 0x7f42fc3c8020: 00060008 0006d7f67598 > 0x7f42fc3c8030: 0006d7f67348 7f43120080d4 > 0x7f42fc3c8040: 0001 0006d7f676d8 > > > SYSTEM CONFIGURATION : > > # CPU - Intel Core i7 - 4771 CPU @ 3.50 GHz * 8 > 2. RAM - 16 GB > 3. STORAGE - 967.6 GB > 4. OS - Ubuntu 16.04 LTS > 5. Apache Cassandra - 2.2.11 > 6. CPP Driver - 2.2.1-1 > 7. libuv - 1.4.2-1 > 8. Java version - "1.8.0_181" > Java (TM) SE Runtime Environment (build 1.8.0_181-b13) > Java Hotspot (TM) 64-Bit Server VM (build 25.181-b13, mixed mode) > > Cassandra was working normally before generating this bug. Now when i am > trying to restart my server getting this bug. > Please help me asap. > > Regards, > Ravi > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To
[jira] [Commented] (CASSANDRA-14805) Fails on running Cassandra server
[
https://issues.apache.org/jira/browse/CASSANDRA-14805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643269#comment-16643269
]
Joshua McKenzie commented on CASSANDRA-14805:
-
{quote}This will not able to help me. Anyother community on Jira or you help me
in this.
Its quite urgent. Please do the needful
{quote}
The community emailing list or IRC are precisely the places you need to go to
if you want to get questions about operationally running the database as
quickly as possible by experts.
> Fails on running Cassandra server
> --
>
> Key: CASSANDRA-14805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14805
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ravi Gangwar
>Priority: Critical
>
> Full Product Version :
> Fails on java version. It's Java "1.8.0_181"
> Os Version : Ubuntu 16.04 LTS
> EXTRA RELEVANT SYSTEM CONFIGURATION :
> Just installed Cassandra 2.2.11
> A DESCRIPTION OF THE PROBLEM :
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGBUS (0x7) at pc=0x7f42fc492e70, pid=12128, tid=0x7f42fc3c9700
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_181-b13) (build
> 1.8.0_181-b13)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.181-b13 mixed mode
> linux-amd64 compressed oops)
> # Problematic frame:
> # C [liblz4-java8098863625230398555.so+0x5e70] LZ4_decompress_fast+0xd0
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
> --- T H R E A D ---
> Current thread (0x006d0800): JavaThread "CompactionExecutor:7" daemon
> [_thread_in_native, id=5278, stack(0x7f42fc389000,0x7f42fc3ca000)]
> siginfo: si_signo: 7 (SIGBUS), si_code: 2 (BUS_ADRERR), si_addr:
> 0x7f2026b8b000
> Registers:
> RAX=0x7f432200750a, RBX=0x7f2026b8affc, RCX=0x7f4322005856,
> RDX=0x7f432200750a
> RSP=0x7f42fc3c7e50, RBP=0x7f2026b8940f, RSI=0x7f4322011d88,
> RDI=0x0007
> R8 =0x7f4322011d84, R9 =0x7f4322011d90, R10=0x0003,
> R11=0x
> R12=0x, R13=0x7f1ffe33d40f, R14=0x7f4322011d87,
> R15=0x7f4322011d8b
> RIP=0x7f42fc492e70, EFLAGS=0x00010287, CSGSFS=0x002b0033,
> ERR=0x0004
> TRAPNO=0x000e
> Top of Stack: (sp=0x7f42fc3c7e50)
> 0x7f42fc3c7e50: 4ce9
> 0x7f42fc3c7e60: 0004 0001
> 0x7f42fc3c7e70: 0002 0001
> 0x7f42fc3c7e80: 0004 0004
> 0x7f42fc3c7e90: 0004 0004
> 0x7f42fc3c7ea0:
> 0x7f42fc3c7eb0:
> 0x7f42fc3c7ec0: 0001
> 0x7f42fc3c7ed0: 0002 0003
> 0x7f42fc3c7ee0: 7f42fc3c7fa8 006d09f8
> 0x7f42fc3c7ef0:
> 0x7f42fc3c7f00: 7f1ffe33d40f 7f4322001d90
> 0x7f42fc3c7f10: 2884c000 7f42fc48f59d
> 0x7f42fc3c7f20: 000733b5fa78
> 0x7f42fc3c7f30: 7f42fc3c7fc0
> 0x7f42fc3c7f40: 2884bfff 7f42fc3c7fa8
> 0x7f42fc3c7f50: 006d0800 7f43119e8f1d
> 0x7f42fc3c7f60: 7f42fc3c7f98
> 0x7f42fc3c7f70: 0001
> 0x7f42fc3c7f80: 00070764f4d8
> 0x7f42fc3c7f90: 00075b5937f0
> 0x7f42fc3c7fa0: 7f42fc3c7ff0 0006f91d2e38
> 0x7f42fc3c7fb0: 0006d7e3f210 7f42fc3c8040
> 0x7f42fc3c7fc0: 00070764f4d8 7f4311f9eca8
> 0x7f42fc3c7fd0: 51070001 00072dc9abb0
> 0x7f42fc3c7fe0: 28851103 0006dd2d
> 0x7f42fc3c7ff0: 00022883f40b eb6b26fe2884bffc
> 0x7f42fc3c8000: 00070764f4d8 00012884c000
> 0x7f42fc3c8010: 00075b5937f0 00075b5937f0
> 0x7f42fc3c8020: 00060008 0006d7f67598
> 0x7f42fc3c8030: 0006d7f67348 7f43120080d4
> 0x7f42fc3c8040: 0001 0006d7f676d8
>
>
> SYSTEM CONFIGURATION :
>
> # CPU - Intel Core i7 - 4771 CPU @ 3.50 GHz * 8
> 2. RAM - 16 GB
> 3. STORAGE - 967.6 GB
> 4. OS - Ubuntu 16.04 LTS
> 5. Apache Cassandra - 2.2.11
> 6. CPP Driver - 2.2.1-1
> 7. libuv - 1.4.2-1
> 8. Java version - "1.8.0_181"
> Java (TM) SE Runtime Environment (build 1.8.0_181-b13)
> Java Hotspot (TM) 64-Bit Server VM (build
[jira] [Resolved] (CASSANDRA-14805) Fails on running Cassandra server
[ https://issues.apache.org/jira/browse/CASSANDRA-14805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie resolved CASSANDRA-14805. - Resolution: Invalid Please reach out to the community on #cassandra on freenode or via the [user mailing lists|[http://cassandra.apache.org/community/].] This Jira is for tracking development of the database. > Fails on running Cassandra server > -- > > Key: CASSANDRA-14805 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14805 > Project: Cassandra > Issue Type: Bug >Reporter: Ravi Gangwar >Priority: Critical > > Full Product Version : > Fails on java version. It's Java "1.8.0_181" > Os Version : Ubuntu 16.04 LTS > EXTRA RELEVANT SYSTEM CONFIGURATION : > Just installed Cassandra 2.2.11 > A DESCRIPTION OF THE PROBLEM : > # > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGBUS (0x7) at pc=0x7f42fc492e70, pid=12128, tid=0x7f42fc3c9700 > # > # JRE version: Java(TM) SE Runtime Environment (8.0_181-b13) (build > 1.8.0_181-b13) > # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.181-b13 mixed mode > linux-amd64 compressed oops) > # Problematic frame: > # C [liblz4-java8098863625230398555.so+0x5e70] LZ4_decompress_fast+0xd0 > # > # Failed to write core dump. Core dumps have been disabled. To enable core > dumping, try "ulimit -c unlimited" before starting Java again > # > # If you would like to submit a bug report, please visit: > # http://bugreport.java.com/bugreport/crash.jsp > # The crash happened outside the Java Virtual Machine in native code. > # See problematic frame for where to report the bug. > # > --- T H R E A D --- > Current thread (0x006d0800): JavaThread "CompactionExecutor:7" daemon > [_thread_in_native, id=5278, stack(0x7f42fc389000,0x7f42fc3ca000)] > siginfo: si_signo: 7 (SIGBUS), si_code: 2 (BUS_ADRERR), si_addr: > 0x7f2026b8b000 > Registers: > RAX=0x7f432200750a, RBX=0x7f2026b8affc, RCX=0x7f4322005856, > RDX=0x7f432200750a > RSP=0x7f42fc3c7e50, RBP=0x7f2026b8940f, RSI=0x7f4322011d88, > RDI=0x0007 > R8 =0x7f4322011d84, R9 =0x7f4322011d90, R10=0x0003, > R11=0x > R12=0x, R13=0x7f1ffe33d40f, R14=0x7f4322011d87, > R15=0x7f4322011d8b > RIP=0x7f42fc492e70, EFLAGS=0x00010287, CSGSFS=0x002b0033, > ERR=0x0004 > TRAPNO=0x000e > Top of Stack: (sp=0x7f42fc3c7e50) > 0x7f42fc3c7e50: 4ce9 > 0x7f42fc3c7e60: 0004 0001 > 0x7f42fc3c7e70: 0002 0001 > 0x7f42fc3c7e80: 0004 0004 > 0x7f42fc3c7e90: 0004 0004 > 0x7f42fc3c7ea0: > 0x7f42fc3c7eb0: > 0x7f42fc3c7ec0: 0001 > 0x7f42fc3c7ed0: 0002 0003 > 0x7f42fc3c7ee0: 7f42fc3c7fa8 006d09f8 > 0x7f42fc3c7ef0: > 0x7f42fc3c7f00: 7f1ffe33d40f 7f4322001d90 > 0x7f42fc3c7f10: 2884c000 7f42fc48f59d > 0x7f42fc3c7f20: 000733b5fa78 > 0x7f42fc3c7f30: 7f42fc3c7fc0 > 0x7f42fc3c7f40: 2884bfff 7f42fc3c7fa8 > 0x7f42fc3c7f50: 006d0800 7f43119e8f1d > 0x7f42fc3c7f60: 7f42fc3c7f98 > 0x7f42fc3c7f70: 0001 > 0x7f42fc3c7f80: 00070764f4d8 > 0x7f42fc3c7f90: 00075b5937f0 > 0x7f42fc3c7fa0: 7f42fc3c7ff0 0006f91d2e38 > 0x7f42fc3c7fb0: 0006d7e3f210 7f42fc3c8040 > 0x7f42fc3c7fc0: 00070764f4d8 7f4311f9eca8 > 0x7f42fc3c7fd0: 51070001 00072dc9abb0 > 0x7f42fc3c7fe0: 28851103 0006dd2d > 0x7f42fc3c7ff0: 00022883f40b eb6b26fe2884bffc > 0x7f42fc3c8000: 00070764f4d8 00012884c000 > 0x7f42fc3c8010: 00075b5937f0 00075b5937f0 > 0x7f42fc3c8020: 00060008 0006d7f67598 > 0x7f42fc3c8030: 0006d7f67348 7f43120080d4 > 0x7f42fc3c8040: 0001 0006d7f676d8 > > > SYSTEM CONFIGURATION : > > # CPU - Intel Core i7 - 4771 CPU @ 3.50 GHz * 8 > 2. RAM - 16 GB > 3. STORAGE - 967.6 GB > 4. OS - Ubuntu 16.04 LTS > 5. Apache Cassandra - 2.2.11 > 6. CPP Driver - 2.2.1-1 > 7. libuv - 1.4.2-1 > 8. Java version - "1.8.0_181" > Java (TM) SE Runtime Environment (build 1.8.0_181-b13) > Java Hotspot (TM) 64-Bit Server VM (build 25.181-b13, mixed mode) > > Cassandra was working normally before generating this bug. Now when i am > trying to restart my server getting this
[jira] [Commented] (CASSANDRA-11935) Add support for arithmetic operators
[
https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16635940#comment-16635940
]
Joshua McKenzie commented on CASSANDRA-11935:
-
Question on a ticket, sure. Re-open a ticket to have that discussion, I'm less
convinced.
We don't have a collective definition of done. You're opening Pandora's Box
when you start advocating for people re-opening tickets that haven't been
'done' to the community's endorsement.
> Add support for arithmetic operators
>
>
> Key: CASSANDRA-11935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11935
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Benjamin Lerer
>Priority: Major
> Fix For: 4.0
>
>
> The goal of this ticket is to add support for arithmetic operators:
> * {{-}}: Change the sign of the argument
> * {{+}}: Addition operator
> * {{-}}: Minus operator
> * {{*}}: Multiplication operator
> * {{/}}: Division operator
> * {{%}}: Modulo operator
> This ticket we should focus on adding operator only for numeric types to keep
> the scope as small as possible. Dates and string operations will be adressed
> in follow up tickets.
> The operation precedence should be:
> # {{*}}, {{/}}, {{%}}
> # {{+}}, {{-}}
> Some implicit data conversion should be performed when operations are
> performed on different types (e.g. double + int).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-11935) Add support for arithmetic operators
[
https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16635912#comment-16635912
]
Joshua McKenzie commented on CASSANDRA-11935:
-
{quote}It's harsh to reopen a ticket that's been quiet for nearly two years
with a justification like _I *personally* prefer this default behaviour_
{quote}
{quote} it's still unreasonable to expect everyone on the project to keep tabs
on everything that happens in every single JIRA and commit. Everyone comes
across issues with changes as they are doing other work and it's fine to
address them as they are discovered.
{quote}
Just as a matter of hygiene and process, I strongly recommend we open new
tickets for these types of post-hoc discussions rather than re-opening tickets.
If everyone re-opened a ticket every time they found something in the code-base
they didn't agree with or wanted to debate, well, that kind of chaos wouldn't
be sustainable.
> Add support for arithmetic operators
>
>
> Key: CASSANDRA-11935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11935
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Benjamin Lerer
>Priority: Major
> Fix For: 4.0
>
>
> The goal of this ticket is to add support for arithmetic operators:
> * {{-}}: Change the sign of the argument
> * {{+}}: Addition operator
> * {{-}}: Minus operator
> * {{*}}: Multiplication operator
> * {{/}}: Division operator
> * {{%}}: Modulo operator
> This ticket we should focus on adding operator only for numeric types to keep
> the scope as small as possible. Dates and string operations will be adressed
> in follow up tickets.
> The operation precedence should be:
> # {{*}}, {{/}}, {{%}}
> # {{+}}, {{-}}
> Some implicit data conversion should be performed when operations are
> performed on different types (e.g. double + int).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14772) Fix issues in audit / full query log interactions
[
https://issues.apache.org/jira/browse/CASSANDRA-14772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626318#comment-16626318
]
Joshua McKenzie commented on CASSANDRA-14772:
-
{quote}I've looked up [the original discussion
thread|https://lists.apache.org/thread.html/61f29a67b4570dac87047f64b846c6f83e8bcb39b6b67143c9a485c2@%3Cdev.cassandra.apache.org%3E],
and the predominant phrasing was "feature freeze"
{quote}
Ah hah - there's our disconnect. I incorrectly recalled us discussing it as a
*code freeze*, not a *feature freeze*.
Two very different things. Refactor away to your heart's content, I won't
fight it. :)
And fwiw - I'm in no way advocating for us to never refactor code; just take a
look at the CommitLogReader during the CDC impl time frame. I'm against
refactors post *code freeze*, which is irrelevant given the earlier
misunderstanding.
Carry on, sorry for the unnecessary traffic.
> Fix issues in audit / full query log interactions
> -
>
> Key: CASSANDRA-14772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14772
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 4.0
>
>
> There are some problems with the audit + full query log code that need to be
> resolved before 4.0 is released:
> * Fix performance regression in FQL that makes it less usable than it should
> be.
> * move full query log specific code to a separate package
> * do some audit log class renames (I keep reading {{BinLogAuditLogger}} vs
> {{BinAuditLogger}} wrong for example)
> * avoid parsing the CQL queries twice in {{QueryMessage}} when audit log is
> enabled.
> * add a new tool to dump audit logs (ie, let fqltool be full query log
> specific). fqltool crashes when pointed to them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14772) Fix issues in audit / full query log interactions
[
https://issues.apache.org/jira/browse/CASSANDRA-14772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626158#comment-16626158
]
Joshua McKenzie commented on CASSANDRA-14772:
-
Opinion presented as fact ftw. :)
There are a variety of ways to interpret "freeze", and we could (and probably
should) take that debate/discussion to the ML.
What I will say here: refactoring code almost always introduces new defects
while fixing old ones; this is a lesson we should all know all too well on this
project.
> Fix issues in audit / full query log interactions
> -
>
> Key: CASSANDRA-14772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14772
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 4.0
>
>
> There are some problems with the audit + full query log code that need to be
> resolved before 4.0 is released:
> * Fix performance regression in FQL that makes it less usable than it should
> be.
> * move full query log specific code to a separate package
> * do some audit log class renames (I keep reading {{BinLogAuditLogger}} vs
> {{BinAuditLogger}} wrong for example)
> * avoid parsing the CQL queries twice in {{QueryMessage}} when audit log is
> enabled.
> * add a new tool to dump audit logs (ie, let fqltool be full query log
> specific). fqltool crashes when pointed to them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14772) Refactor audit / full query log
[
https://issues.apache.org/jira/browse/CASSANDRA-14772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625898#comment-16625898
]
Joshua McKenzie commented on CASSANDRA-14772:
-
Having not looked at the code, is there value in moving the API changes and
defect fixes to their own ticket targeting 4.0 and doing the other parts 4.0.x
(black box refactors, etc)?
I'm mostly coming at this from a "broken windows theory" angle of us having
discipline w/the concept of a code-freeze. Don't want us to get into the habit
of sneaking things in past the date, which then further risks destabilizing the
testing period.
> Refactor audit / full query log
> ---
>
> Key: CASSANDRA-14772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14772
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 4.0
>
>
> I find the audit + full query log code quite hard to follow with some of the
> naming and mixing of concepts, this ticket aims to:
> * move full query log specific code to a separate package
> * do some audit log class renames (I keep reading {{BinLogAuditLogger}} vs
> {{BinAuditLogger}} wrong for example)
> * avoid parsing the CQL queries twice in {{QueryMessage}} when audit log is
> enabled.
> * add a new tool to dump audit logs (ie, let fqltool be full query log
> specific)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14772) Refactor audit / full query log
[
https://issues.apache.org/jira/browse/CASSANDRA-14772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625857#comment-16625857
]
Joshua McKenzie commented on CASSANDRA-14772:
-
I wouldn't consider a refactor to be a bug-fix. While there's a small minority
of this ticket that's defect-related, I think committing a refactor post-freeze
date for 4.0 is not good hygiene.
Am I missing something here?
> Refactor audit / full query log
> ---
>
> Key: CASSANDRA-14772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14772
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 4.0
>
>
> I find the audit + full query log code quite hard to follow with some of the
> naming and mixing of concepts, this ticket aims to:
> * move full query log specific code to a separate package
> * do some audit log class renames (I keep reading {{BinLogAuditLogger}} vs
> {{BinAuditLogger}} wrong for example)
> * avoid parsing the CQL queries twice in {{QueryMessage}} when audit log is
> enabled.
> * add a new tool to dump audit logs (ie, let fqltool be full query log
> specific)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14621) Refactor CompactionStrategyManager
[ https://issues.apache.org/jira/browse/CASSANDRA-14621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14621: Fix Version/s: (was: 4.x) 4.0 > Refactor CompactionStrategyManager > -- > > Key: CASSANDRA-14621 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14621 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Minor > Fix For: 4.0 > > > CompactionStrategyManager grew a decent amount of duplicated code as part of > CASSANDRA-9143, which added pendingRepairs alongside the repaired and > unrepaired buckets. At this point, the logic that routes sstables between the > different buckets, and the different partition range divisions has gotten a > little complex, and editing it is tedious and error prone. With transient > replication requiring yet another bucket for, this seems like a good time to > split some of the functionality of CSM into other classes, and make sstable > routing a bit more generalized. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14570) Improper default value of cdc_total_space_in_mb
[
https://issues.apache.org/jira/browse/CASSANDRA-14570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546991#comment-16546991
]
Joshua McKenzie commented on CASSANDRA-14570:
-
A couple concerns:
# If your volume for CDC logs doesn't have available space left for
{code:java}
size + conf.commitlog_total_space_in_mb{code}
, assuming size 0 and cl size > free_space on volume, this patch will still set
that value to our cdc_total in mb which would be problematic. We should take
into consideration whether size + cl size is > the estimated value of the
volume and consider the best way to handle that error there (be it warn, fail
startup, respect CL failure policy, etc)
# This is now a misleading property name since cdc storage space will now be
"desired cdc 'buffer' plus cl size on disk". It would probably make sense (and
be more user friendly) to rename \{cdc_total_space_in_mb} to
\{cdc_extra_buffer_space} and comment that param to mention the relationship
between it and the CL space. Probably also want to keep respecting and using
the old property name for backwards compat. purposes.
That being said, great catch on those silly / nonsensical defaults, problem
w/CDC post hard-linking.
> Improper default value of cdc_total_space_in_mb
> ---
>
> Key: CASSANDRA-14570
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14570
> Project: Cassandra
> Issue Type: Bug
> Components: Configuration
>Reporter: Shichao An
>Assignee: Shichao An
>Priority: Minor
> Labels: CDC
>
> The code for calculating cdc_total_space_in_mb in DatabaseDescriptor.java
> does not reflect the latest architecture introduced by CASSANDRA-12148.
> In short, cdc_total_space_in_mb should be equal or greater than
> commitlog_total_space_in_mb; otherwise, the writes will fail when on-disk
> commit logs size reaches the value of cdc_total_space_in_mb. For example, If
> cdc_total_space_in_mb is 4096 and commitlog_total_space_in_mb is 8192, when
> we enabled the cdc_enabled flag (even if we didn't enable cdc=true on any
> table), when total size of commit logs reaches 4096 MB, there is the same of
> amount of hard links in cdc_raw directory, that is, 4096 MB of cdc logs. Then
> the CommitLogSegmentManagerCDC will be unable to process new CL segments.
> (See CommitLogSegmentManagerCDC.processNewSegment method)
>
> I will attach patches later.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14295) no ssl hostname validation in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-14295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14295:
Priority: Minor (was: Major)
> no ssl hostname validation in cqlsh
> ---
>
> Key: CASSANDRA-14295
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14295
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Christian Becker
>Priority: Minor
> Labels: Security
>
> In order to validate certificates properly the python driver requires
> {{check_hostname}} to be set.
> [https://github.com/datastax/python-driver/blob/master/cassandra/cluster.py#L558-L562]
> However it is not available as a setting in cqlsh:
> [https://github.com/apache/cassandra/blob/trunk/pylib/cqlshlib/sslhandling.py#L86-L89]
> I noticed this because cqlsh is connecting to 127.0.0.1 per default, but the
> configured certificate is just containing the hostname and the local ip. The
> connection was always successful. But when adding {{check_hostname}} to
> {{cqlshlib/sslhandling.py}} the validation works as expected:
> current behaviour:
> {code:java}
> # cqlsh --ssl
> Connected to -cassandra at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.11.2 | CQL spec 3.4.4 | Native protocol v4]
> Use HELP for help.
> @cqlsh>{code}
> expected:
> {code:java}
> # cqlsh --ssl
> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
> CertificateError("hostname '127.0.0.1' doesn't match ''",)}){code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14300) Nodetool upgradesstables erring out with Null assertion error (2.2.5 to 3.11.1)
[ https://issues.apache.org/jira/browse/CASSANDRA-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14300: Priority: Major (was: Blocker) > Nodetool upgradesstables erring out with Null assertion error (2.2.5 to > 3.11.1) > --- > > Key: CASSANDRA-14300 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14300 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Bhanu M. Gandikota >Priority: Major > Attachments: apache-cassandra-modified-3.11.1.jar > > > -bash-4.2$ nodetool upgradesstables > > WARN 11:28:28,430 Small cdc volume detected at /cdc_raw; setting > cdc_total_space_in_mb to 1982. You can override this in cassandra.yaml > > error: null > -- StackTrace -- > java.lang.AssertionError > at org.apache.cassandra.db.rows.BufferCell.(BufferCell.java:43) > at > org.apache.cassandra.db.LegacyLayout$CellGrouper.addCell(LegacyLayout.java:1242) > at > org.apache.cassandra.db.LegacyLayout$CellGrouper.addAtom(LegacyLayout.java:1185) > at > org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.readRow(UnfilteredDeserializer.java:495) > at > org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:472) > at > org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:306) > at > org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.readStaticRow(SSTableSimpleIterator.java:176) > at > org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:49) > at > org.apache.cassandra.io.sstable.SSTableIdentityIterator.create(SSTableIdentityIterator.java:59) > at > org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator$1.initializeIterator(BigTableScanner.java:384) > at > org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48) > at > org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:70) > at > org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:122) > at > org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:113) > at > org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:466) > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > at > org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:163) > at > org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:92) > at > org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:233) > at > org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:196) > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at > org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:85) > at > org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61) > at > org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:428) > at > org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:315) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at > org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81) > at java.lang.Thread.run(Thread.java:745) > > -bash-4.2$ -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14295) no ssl hostname validation in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-14295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14295:
Component/s: CQL
> no ssl hostname validation in cqlsh
> ---
>
> Key: CASSANDRA-14295
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14295
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Christian Becker
>Priority: Major
> Labels: Security
>
> In order to validate certificates properly the python driver requires
> {{check_hostname}} to be set.
> [https://github.com/datastax/python-driver/blob/master/cassandra/cluster.py#L558-L562]
> However it is not available as a setting in cqlsh:
> [https://github.com/apache/cassandra/blob/trunk/pylib/cqlshlib/sslhandling.py#L86-L89]
> I noticed this because cqlsh is connecting to 127.0.0.1 per default, but the
> configured certificate is just containing the hostname and the local ip. The
> connection was always successful. But when adding {{check_hostname}} to
> {{cqlshlib/sslhandling.py}} the validation works as expected:
> current behaviour:
> {code:java}
> # cqlsh --ssl
> Connected to -cassandra at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.11.2 | CQL spec 3.4.4 | Native protocol v4]
> Use HELP for help.
> @cqlsh>{code}
> expected:
> {code:java}
> # cqlsh --ssl
> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
> CertificateError("hostname '127.0.0.1' doesn't match ''",)}){code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14259) Nodetool command is not accessible with read-only account.
[ https://issues.apache.org/jira/browse/CASSANDRA-14259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14259: Component/s: Tools > Nodetool command is not accessible with read-only account. > -- > > Key: CASSANDRA-14259 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14259 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Kamil >Priority: Major > > In production environment it's important to have comprehensive cluster > monitoring. That's where nodetool comes handy. However for security reason > tech ops staff gets read-only account and with that account they are unable > to pull basics stats like cfstats etc. There is an option to use Rest API but > due to security and authentication issues this is not always a valid option. > Therefore we need to be able to log in to nodetool with RO account and be > able to pass aruments which do not change state of Cassandra cluster. > > Another improvement would be to have access to nodetool functions within > CQLSH. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14012) Document gossip protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-14012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14012: Component/s: Documentation and Website > Document gossip protocol > > > Key: CASSANDRA-14012 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14012 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Jörn Heissler >Priority: Minor > Labels: Documentation > > I had an issue today with two nodes communicating with each other; there's a > flaw in my configuration (wrong broadcast address). > I saw a little bit of traffic on port 7000, but I couldn't understand it for > lack of documentation. > With documentation I would have understood my issue very quickly (7f 00 01 01 > is a bad broadcast address!). But I didn't recognize those 4 bytes as the bc > address. > Could you please document the gossip protocol? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14011) Multi threaded L0 -> L1 compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-14011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14011:
Component/s: Compaction
> Multi threaded L0 -> L1 compaction
> --
>
> Key: CASSANDRA-14011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14011
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
>Reporter: Marcus Eriksson
>Priority: Major
> Fix For: 4.x
>
>
> Currently L0 -> L1 compactions are almost always single threaded because
> every L0 sstable will overlap with all L1 sstables. To improve this, we
> should range-split the input sstables in a configurable amount of parts and
> then use multiple threads to write out the results. This is similar to the
> {{max_subcompactions}} option in RocksDB:
> https://github.com/facebook/rocksdb/wiki/Leveled-Compaction
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13990) Remove OldNetworkTopologyStrategy
[ https://issues.apache.org/jira/browse/CASSANDRA-13990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-13990: Labels: lhf (was: ) > Remove OldNetworkTopologyStrategy > - > > Key: CASSANDRA-13990 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13990 > Project: Cassandra > Issue Type: Wish > Components: Configuration >Reporter: Jeremy Hanna >Priority: Trivial > Labels: lhf > > RackAwareStrategy was renamed OldNetworkTopologyStrategy back in 0.7 > (CASSANDRA-1392) and it's still around. Is there any reason to keep this > relatively dead code in the codebase at this point? I'm not aware of its use > and it sometimes confuses users. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13957) upgradesstables fails after upgrading from 2.1.x to 3.0.14
[
https://issues.apache.org/jira/browse/CASSANDRA-13957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539100#comment-16539100
]
Joshua McKenzie commented on CASSANDRA-13957:
-
[~dan.priscornic]: did you ever get this resolved?
> upgradesstables fails after upgrading from 2.1.x to 3.0.14
> --
>
> Key: CASSANDRA-13957
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13957
> Project: Cassandra
> Issue Type: Bug
>Reporter: Dan Priscornic
>Priority: Major
>
> After upgrading DSE from 4.8.14 (cassandra 2.1.18.1463) to 5.0.10 (cassandra
> 3.0.14.1862) I ran nodetool upgradesstables and it fails with the following
> stack trace:
> {code:java}
> # nodetool -u cassandra -pwf /etc/dse/cassandra/jmxremote.password
> upgradesstables
> error: null
> -- StackTrace --
> java.lang.AssertionError
> at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:137)
> at
> org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:111)
> at
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:57)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:205)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:99)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:427)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:314)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$6/61137731.run(Unknown
> Source)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> The bug seems similar to CASSANDRA-13320 which says it should be fixed in
> cassandra 3.0.13 but does not look fixed in 3.0.14
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-13946) Updating row TTL without updating values
[ https://issues.apache.org/jira/browse/CASSANDRA-13946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie resolved CASSANDRA-13946. - Resolution: Incomplete Closing - feel free to add a description and re-open. > Updating row TTL without updating values > > > Key: CASSANDRA-13946 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13946 > Project: Cassandra > Issue Type: New Feature > Components: Core, CQL >Reporter: Tomer >Priority: Trivial > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13932) Stress write order and seed order should be different
[
https://issues.apache.org/jira/browse/CASSANDRA-13932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-13932:
Component/s: Stress
> Stress write order and seed order should be different
> -
>
> Key: CASSANDRA-13932
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13932
> Project: Cassandra
> Issue Type: Bug
> Components: Stress, Tools
>Reporter: Daniel Cranford
>Priority: Major
> Labels: stress
> Attachments: 0001-Initial-implementation-cassandra-3.11.patch,
> vmtouch-after.txt, vmtouch-before.txt
>
>
> Read tests get an unrealistic boost in performance because they read data
> from a set of partitions that was written sequentially.
> I ran into this while running a timed read test against a large data set (250
> million partition keys) {noformat}cassandra-stress read
> duration=30m{noformat} While the test was running, I noticed one node was
> performing zero IO after an initial period.
> I discovered each node in the cluster only had blocks from a single SSTable
> loaded in the FS cache. {noformat}vmtouch -v /path/to/sstables{noformat}
> For the node that was performing zero IO, the SSTable in question was small
> enough to fit into the FS cache.
> I realized that when a read test is run for a duration or until rate
> convergenge, the default population for the seeds is a GAUSSIAN distribution
> over the first million seeds. Because of the way compaction works, partitions
> that are written sequentially will (with high probability) always live in the
> same SSTable. That means that while the first million seeds will generate
> partition keys that will be randomly distributed in the token space, they
> will most likely all live in the same SSTable. When this SSTable is small
> enough to fit into the FS cache, you get unbelievably good results for a read
> test. Consider that a dataset 4x the size of the FS cache will have almost
> 1/2 the data in SSTables small enough to fit into the FS cache.
> Adjusting the population of seeds used during the read test to be the entire
> 250 million seeds used to load the cluster does not fix the
> problem.{noformat}cassandra-stress read duration=30m -pop
> dist=gaussian(1..250M){noformat}
> or (same population, larger sample) {noformat}cassandra-stress read
> n=250M{noformat}
> Any distribution other than the uniform distribution has one or more modes,
> and the mode(s) of such a distribution will cluster reads around a certain
> seed range which corresponds to a certain set of sequential writes which
> corresponds to (with high probability) a single SSTable.
> My patch against cassandra-3.11 fixes this by shuffling the sequence of
> generated seeds. Each seed value will still be generated once and only once.
> The old behavior of sequential seed generation (ie seed(n+1) = seed( n) + 1)
> may be selected by using the no-shuffle flag. e.g. {noformat}cassandra-stress
> read duration=30m -pop no-shuffle{noformat}
> Results: In [^vmtouch-before.txt] only pages from a single SSTable are
> present in the FS cache while in [^vmtouch-after.txt] an equal proportion of
> all SSTables are present in the FS cache.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13920) Adding regular column to COMPACT STORAGE with w/o clustering keys table causes an exception
[
https://issues.apache.org/jira/browse/CASSANDRA-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-13920:
Component/s: CQL
Core
> Adding regular column to COMPACT STORAGE with w/o clustering keys table
> causes an exception
> ---
>
> Key: CASSANDRA-13920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13920
> Project: Cassandra
> Issue Type: Bug
> Components: Core, CQL
>Reporter: Alex Petrov
>Priority: Minor
>
> When trying to {{ALTER}} a {{COMPACT TABLE}} without clustering keys (i.e.
> non-dense one), you'll get an exception.
> Adding regular columns to non-dense compact tables should be forbidden
> (adding static ones already fails with {{Static columns are not allowed in
> COMPACT STORAGE tables}}), just as adding columns to dense compact tables
> which throws {{Cannot add new column to a COMPACT STORAGE table}} (or the
> error message should be adjusted).
> dtest to reproduce:
> {code}
> from cql_tests import CQLTester
> class StorageProxyCQLTester(CQLTester):
> def test_sparse_compact(self):
> session = self.prepare(nodes=2, rf=2)
> session.execute("CREATE TABLE sparse_compact_table (k int PRIMARY
> KEY, v1 int, v2 int) WITH COMPACT STORAGE;")
> session.execute("ALTER TABLE sparse_compact_table ADD wat int",)
> {code}
> Exception:
> {code}
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.CompactTables.getCompactValueColumn(CompactTables.java:67)
> ~[main/:na]
> at
> org.apache.cassandra.config.CFMetaData.rebuild(CFMetaData.java:337)
> ~[main/:na]
> at
> org.apache.cassandra.config.CFMetaData.validate(CFMetaData.java:935)
> ~[main/:na]
> at
> org.apache.cassandra.service.MigrationManager.announceColumnFamilyUpdate(MigrationManager.java:421)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.statements.AlterTableStatement.announceMigration(AlterTableStatement.java:288)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:93)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222)
> ~[main/:na]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
> ~[main/:na]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513)
> [main/:na]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407)
> [main/:na]
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:348)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_121]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> [main/:na]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13917) COMPACT STORAGE inserts on tables without clusterings accept hidden column1 and value columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-13917:
Component/s: Core
> COMPACT STORAGE inserts on tables without clusterings accept hidden column1
> and value columns
> -
>
> Key: CASSANDRA-13917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13917
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Alex Petrov
>Assignee: Aleksandr Sorokoumov
>Priority: Minor
> Labels: lhf
> Fix For: 3.0.x, 3.11.x
>
>
> Test for the issue:
> {code}
> @Test
> public void testCompactStorage() throws Throwable
> {
> createTable("CREATE TABLE %s (a int PRIMARY KEY, b int, c int) WITH
> COMPACT STORAGE");
> assertInvalid("INSERT INTO %s (a, b, c, column1) VALUES (?, ?, ?,
> ?)", 1, 1, 1, ByteBufferUtil.bytes('a'));
> // This one fails with Some clustering keys are missing: column1,
> which is still wrong
> assertInvalid("INSERT INTO %s (a, b, c, value) VALUES (?, ?, ?, ?)",
> 1, 1, 1, ByteBufferUtil.bytes('a'));
> assertInvalid("INSERT INTO %s (a, b, c, column1, value) VALUES (?, ?,
> ?, ?, ?)", 1, 1, 1, ByteBufferUtil.bytes('a'), ByteBufferUtil.bytes('b'));
> assertEmpty(execute("SELECT * FROM %s"));
> }
> {code}
> Gladly, these writes are no-op, even though they succeed.
> {{value}} and {{column1}} should be completely hidden. Fixing this one should
> be as easy as just adding validations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13908) Add JDK9 new CRC32C checksum
[ https://issues.apache.org/jira/browse/CASSANDRA-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-13908: Labels: Performance (was: ) > Add JDK9 new CRC32C checksum > > > Key: CASSANDRA-13908 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13908 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Yingqi Lu >Priority: Major > Labels: Performance > > Dear All, > JDK9 introduces a new API java.util.zip.CRC32C. Our early data show CRC32C > provides additional performance benefits over CRC32 checksum. > We are proposing to add CRC32C as one of the checksum types in addition to > Adler32 and CRC32. Performance data and prototype code changes will be > submitted here soon. At the meantime, please feel free to provide your > feedback and comments. > Thanks, > Yingqi Lu -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13903) Add JDK9 vectorizedMismatch array compare support
[ https://issues.apache.org/jira/browse/CASSANDRA-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-13903: Labels: Performance (was: ) > Add JDK9 vectorizedMismatch array compare support > - > > Key: CASSANDRA-13903 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13903 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Yingqi Lu >Priority: Major > Labels: Performance > > Dear All, > JDK9 has recently been released (http://openjdk.java.net/projects/jdk9/). It > introduces a set of new vectorizedMismatch APIs for array comparison. On > supported platforms, the new JDK9 implementation is intrisified to leverage > SIMD instructions. For a byte array comparison, up to 64 bytes (512 bits) can > be compared as a single unit. Feature details please refer to > http://download.java.net/java/jdk9/docs/api/java/util/Arrays.html and > https://bugs.openjdk.java.net/browse/JDK-8033148 > Currently in Cassandra, keys are implemented as ByteBuffers and compared as > byte arrays most of the time. Key comparison, for example decorated key > compare, it is either done with unsafe operation taking 8 bytes at a time, or > pure java operation comparing byte by byte. This can be optimized with new > JDK9 java.util.Arrays.compare APIs on modern CPUs. > Please let us know your feedback. At the meantime, we will submit a patch > with performance studies in couple of weeks. > Thanks, > Yingqi Lu -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13887) Add SASI metrics to JMX
[
https://issues.apache.org/jira/browse/CASSANDRA-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-13887:
Component/s: Observability
> Add SASI metrics to JMX
> ---
>
> Key: CASSANDRA-13887
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13887
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability, sasi
>Reporter: James Howe
>Priority: Minor
>
> Currently there are MBeans for secondary index metrics
> {{org.apache.cassandra.metrics:type=IndexTable}} but I cannot see SASI
> metrics anywhere.
> The only place they're even mentioned is in the table's {{BuiltIndexes}} list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14242) Indexed static column returns inconsistent results
[
https://issues.apache.org/jira/browse/CASSANDRA-14242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14242:
Component/s: Secondary Indexes
> Indexed static column returns inconsistent results
> --
>
> Key: CASSANDRA-14242
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14242
> Project: Cassandra
> Issue Type: Bug
> Components: Secondary Indexes
> Environment: Cassandra 3.11.2
> Java driver 3.4.0
> Ubuntu - 4.4.0-112-generic
>Reporter: Ross Black
>Assignee: Andrés de la Peña
>Priority: Major
> Fix For: 3.0.x, 3.11.x
>
>
> I am using Cassandra 3.11.2, and the Java driver 3.4.0
> I have a table that has a static column, where the static column has a
> secondary index.
> When querying the table I get incomplete or duplicated results, depending on
> the fetch size.
> e.g.
> {code:java}
> CREATE KEYSPACE hack WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> CREATE TABLE hack.stuff (id int, kind text, chunk int static, val1 int,
> PRIMARY KEY (id, kind));
> CREATE INDEX stuff_chunk_index ON hack.stuff (chunk);{code}
> -- repeat with thousands of values for id =>
> {code:java}
> INSERT INTO hack.stuff (id, chunk, kind, val1 ) VALUES (${id}, 777, 'A',
> 123);{code}
> Querying from Java:
> {code:java}
> final SimpleStatement statement = new SimpleStatement("SELECT id, kind,
> val1 FROM hack.stuff WHERE chunk = " + chunk);
> statement.setFetchSize(fetchSize);
> statement.setConsistencyLevel(ConsistencyLevel.ALL);
> final ResultSet resultSet = connection.getSession().execute(statement);
> for (Row row : resultSet) {
> final int id = row.getInt("id");
> }{code}
> *The number of results returned depends on the fetch-size.*
> e.g. For 30k values inserted, I get the following:
> ||fetch-size||result-size||
> |4|3|
> |2|30001|
> |5000|30006|
> |100|30303|
> In production, I have a much larger table where the correct result size for a
> specific chunk is 20019, but some fetch sizes will return _significantly
> fewer_ results.
> ||fetch-size||result-size|| ||
> |25000|20019| |
> |5000||*<== this one is has far fewer results*|
> |5001|20026| |
> (so far been unable to reproduce this with the simpler test table)
> Thanks,
> Ross
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14239) OutOfMemoryError when bootstrapping with less than 100GB RAM
[ https://issues.apache.org/jira/browse/CASSANDRA-14239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14239: Labels: (was: materializedviews) > OutOfMemoryError when bootstrapping with less than 100GB RAM > > > Key: CASSANDRA-14239 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14239 > Project: Cassandra > Issue Type: Bug > Components: Materialized Views > Environment: Details of the bootstrapping Node > * ProLiant BL460c G7 > * 56GB RAM > * 2x 146GB 10K HDD (One dedicated for Commitlog, one for Data, Hints and > saved_caches) > * CentOS 7.4 on SD-Card > * /tmp and /var/log on tmpfs > * Oracle JDK 1.8.0_151 > * Cassandra 3.11.1 > Cluster > * 10 existing Nodes (Up and Normal) >Reporter: Jürgen Albersdorfer >Priority: Major > Attachments: Objects-by-class.csv, > Objects-with-biggest-retained-size.csv, Selection_420.png, Selection_421.png, > cassandra-env.sh, cassandra.yaml, dstat.png, gc.log.0.201804111524.zip, > gc.log.0.current.zip, gc.log.20180441.zip, jvm.options, jvm_opts.txt, > stack-traces.txt > > > Hi, I face an issue when bootstrapping a Node having less than 100GB RAM on > our 10 Node C* 3.11.1 Cluster. > During bootstrap, when I watch the cassandra.log I observe a growth in JVM > Heap Old Gen which gets not significantly freed up any more. > I know that JVM collects on Old Gen only when really needed. I can see > collections, but there is always a remainder which seems to grow forever > without ever getting freed. > After the Node successfully Joined the Cluster, I can remove the extra RAM I > have given it for bootstrapping without any further effect. > It feels like Cassandra will not forget about every single byte streamed over > the Network over time during bootstrapping, - which would be a memory leak > and a major problem, too. > I was able to produce a HeapDumpOnOutOfMemoryError from a 56GB Node (40 GB > assigned JVM Heap). YourKit Profiler shows huge amount of Memory allocated > for org.apache.cassandra.db.Memtable (22 GB) > org.apache.cassandra.db.rows.BufferCell (19 GB) and java.nio.HeapByteBuffer > (11 GB) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14239) OutOfMemoryError when bootstrapping with less than 100GB RAM
[ https://issues.apache.org/jira/browse/CASSANDRA-14239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14239: Component/s: Materialized Views > OutOfMemoryError when bootstrapping with less than 100GB RAM > > > Key: CASSANDRA-14239 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14239 > Project: Cassandra > Issue Type: Bug > Components: Materialized Views > Environment: Details of the bootstrapping Node > * ProLiant BL460c G7 > * 56GB RAM > * 2x 146GB 10K HDD (One dedicated for Commitlog, one for Data, Hints and > saved_caches) > * CentOS 7.4 on SD-Card > * /tmp and /var/log on tmpfs > * Oracle JDK 1.8.0_151 > * Cassandra 3.11.1 > Cluster > * 10 existing Nodes (Up and Normal) >Reporter: Jürgen Albersdorfer >Priority: Major > Attachments: Objects-by-class.csv, > Objects-with-biggest-retained-size.csv, Selection_420.png, Selection_421.png, > cassandra-env.sh, cassandra.yaml, dstat.png, gc.log.0.201804111524.zip, > gc.log.0.current.zip, gc.log.20180441.zip, jvm.options, jvm_opts.txt, > stack-traces.txt > > > Hi, I face an issue when bootstrapping a Node having less than 100GB RAM on > our 10 Node C* 3.11.1 Cluster. > During bootstrap, when I watch the cassandra.log I observe a growth in JVM > Heap Old Gen which gets not significantly freed up any more. > I know that JVM collects on Old Gen only when really needed. I can see > collections, but there is always a remainder which seems to grow forever > without ever getting freed. > After the Node successfully Joined the Cluster, I can remove the extra RAM I > have given it for bootstrapping without any further effect. > It feels like Cassandra will not forget about every single byte streamed over > the Network over time during bootstrapping, - which would be a memory leak > and a major problem, too. > I was able to produce a HeapDumpOnOutOfMemoryError from a 56GB Node (40 GB > assigned JVM Heap). YourKit Profiler shows huge amount of Memory allocated > for org.apache.cassandra.db.Memtable (22 GB) > org.apache.cassandra.db.rows.BufferCell (19 GB) and java.nio.HeapByteBuffer > (11 GB) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14220) C* returns to the driver NullPointerException sometimes when creating a keyspace
[
https://issues.apache.org/jira/browse/CASSANDRA-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14220:
Component/s: Distributed Metadata
Core
> C* returns to the driver NullPointerException sometimes when creating a
> keyspace
> -
>
> Key: CASSANDRA-14220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14220
> Project: Cassandra
> Issue Type: Bug
> Components: Core, Distributed Metadata
> Environment: C* version: 3.11.1
>Reporter: Jaume M
>Assignee: Jaydeepkumar Chovatia
>Priority: Major
> Attachments: node1_system.log, node2_system.log, node3_system.log
>
>
> This happening regular on the python driver CI and it's reasonable to assume
> that C* is under heavy load when it happens. The CQL command sent is
> {code}
> CREATE KEYSPACE keyspace WITH replication " = {'class': 'SimpleStrategy',
> 'replication_factor': 1}
> {code}
> or
> {code}
> CREATE TABLE table_name (k INT, i INT, PRIMARY KEY(k, i))
> {code}
> The error the driver gets is
> {code}
> message="java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> java.lang.NullPointerException">
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14204) Nodetool garbagecollect AssertionError
[
https://issues.apache.org/jira/browse/CASSANDRA-14204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14204:
Component/s: Compaction
> Nodetool garbagecollect AssertionError
> --
>
> Key: CASSANDRA-14204
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14204
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Vincent White
>Assignee: Vincent White
>Priority: Minor
> Fix For: 3.11.x, 4.x
>
>
> When manually running a garbage collection compaction across a table with
> unrepaired sstables and only_purge_repaired_tombstones set to true an
> assertion error is thrown. This is because the unrepaired sstables aren't
> being removed from the transaction as they are filtered out in
> filterSSTables().
> ||3.11||trunk||
> |[branch|https://github.com/vincewhite/cassandra/commit/e13c822736edd3df3403c02e8ef90816f158cde2]|[branch|https://github.com/vincewhite/cassandra/commit/cc8828576404e72504d9b334be85f84c90e77aa7]|
> The stacktrace:
> {noformat}
> -- StackTrace --
> java.lang.AssertionError
> at
> org.apache.cassandra.db.compaction.CompactionManager.parallelAllSSTableOperation(CompactionManager.java:339)
> at
> org.apache.cassandra.db.compaction.CompactionManager.performGarbageCollection(CompactionManager.java:476)
> at
> org.apache.cassandra.db.ColumnFamilyStore.garbageCollect(ColumnFamilyStore.java:1579)
> at
> org.apache.cassandra.service.StorageService.garbageCollect(StorageService.java:3069)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
[jira] [Updated] (CASSANDRA-14200) NullPointerException when dumping sstable with null value for timestamp column
[
https://issues.apache.org/jira/browse/CASSANDRA-14200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14200:
Component/s: Core
> NullPointerException when dumping sstable with null value for timestamp column
> --
>
> Key: CASSANDRA-14200
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14200
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Major
> Fix For: 3.0.x
>
>
> We have an sstable whose schema has a column of type timestamp and it's not
> part of primary key. When dumping the sstable using sstabledump there is NPE
> like this:
> {code:java}
> Exception in thread "main" java.lang.NullPointerException
> at java.util.Calendar.setTime(Calendar.java:1770)
> at java.text.SimpleDateFormat.format(SimpleDateFormat.java:943)
> at java.text.SimpleDateFormat.format(SimpleDateFormat.java:936)
> at java.text.DateFormat.format(DateFormat.java:345)
> at
> org.apache.cassandra.db.marshal.TimestampType.toJSONString(TimestampType.java:93)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:442)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:376)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:280)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:215)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:104)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242){code}
> The reason is that we use a null Date when there is no value for this column:
> {code}
> public Date deserialize(ByteBuffer bytes)
> {
> return bytes.remaining() == 0 ? null : new
> Date(ByteBufferUtil.toLong(bytes));
> }
> {code}
> It seems that we should not deserialize columns with null values.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14179) Corrections in native_protocol_v4.spec
[ https://issues.apache.org/jira/browse/CASSANDRA-14179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14179: Component/s: Documentation and Website > Corrections in native_protocol_v4.spec > -- > > Key: CASSANDRA-14179 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14179 > Project: Cassandra > Issue Type: Bug > Components: Documentation and Website >Reporter: Daragan Bogdan >Priority: Minor > Labels: documentation, easyfix, pull-request-available > Fix For: 3.11.x > > Original Estimate: 0.25h > Remaining Estimate: 0.25h > > Fixing inaccuracy in documentation for native_protocol_v4.spec > Pull reques [link|https://github.com/apache/cassandra/pull/185]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14176) Cannot export & import data containing no-break space (U+00A0)
[
https://issues.apache.org/jira/browse/CASSANDRA-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14176:
Component/s: Tools
> Cannot export & import data containing no-break space (U+00A0)
> --
>
> Key: CASSANDRA-14176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14176
> Project: Cassandra
> Issue Type: Bug
> Components: Libraries, Tools
>Reporter: Marcel Dopita
>Assignee: Marcel Dopita
>Priority: Major
> Attachments: fix.patch
>
>
> We were unable to export and then import the same data to Cassandra - like
> line breaks or the no-break space (U+00A0).
> Adding v.decode() to copyutil.py fixed most characters like line breaks etc.
> Only after using the included patch, the character U+00A0 was correctly
> stored in Cassandra and successfully (verifiable) exported & imported &
> exported.
>
> {code:java}
> diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
> index 7f97b49..883c957 100644
> --- a/pylib/cqlshlib/copyutil.py
> +++ b/pylib/cqlshlib/copyutil.py
> @@ -1871,7 +1871,7 @@ class ImportConversion(object):
> return bytearray.fromhex(v[2:])
>
> def convert_text(v, **_):
> - return v
> + return v.decode('string_escape')
>
> def convert_uuid(v, **_):
> return UUID(v)
> diff --git a/pylib/cqlshlib/formatting.py b/pylib/cqlshlib/formatting.py
> index 803ea63..79eb691 100644
> --- a/pylib/cqlshlib/formatting.py
> +++ b/pylib/cqlshlib/formatting.py
> @@ -33,7 +33,7 @@ from util import UTC
>
> is_win = platform.system() == 'Windows'
>
> -unicode_controlchars_re = re.compile(r'[\x00-\x31\x7f-\xa0]')
> +unicode_controlchars_re = re.compile(r'[\x00-\x31]')
> controlchars_re = re.compile(r'[\x00-\x31\x7f-\xff]')
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14129) BatchStatement (LOGGED BATCH) fails: Not enough replica available for query at consistency ONE (1 required but only 0 alive)
[ https://issues.apache.org/jira/browse/CASSANDRA-14129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14129: Component/s: Coordination > BatchStatement (LOGGED BATCH) fails: Not enough replica available for query > at consistency ONE (1 required but only 0 alive) > > > Key: CASSANDRA-14129 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14129 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Erik Scheuter >Priority: Major > > In our test-cluster I get the error "Not enough replica available for query > at consistency ONE (1 required but only 0 alive)" when 1 node is down and a > BatchStatement is used. By default it uses LOGGED BATCHES and > DCAwareRoundRobinPolicy. > We have 2 DC's, 4 nodes and a RF 2:2. Write Consistency of QUORUM (*not* > LOCAL QUORUM). > I noticed as soon as the driver marked the LocalDC containing the one with > the node down, this error occurs. If the other DC is marked as the LocalDC, > everything works as expected. > The issue https://issues.apache.org/jira/browse/CASSANDRA-7870 points to the > comment from Olivier Michallat at > [StackOverflow|https://stackoverflow.com/questions/26015282/cassandra-not-enough-replica-available-java-driver-behaviour-different-from-c]: > _This batch log must be replicated to at least one replica other than the > coordinator, otherwise that would defeat the above mechanism._ > I think it needs an other replica in the same DC. > These constraints can lead to downtime, so I expect them to find in the > documentation (and a more detailed error message). > Used: > Cassandra 2.0.17 > Datastax-driver 3.3.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14103) Fix potential race during compaction strategy reload
[
https://issues.apache.org/jira/browse/CASSANDRA-14103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14103:
Component/s: Compaction
> Fix potential race during compaction strategy reload
>
>
> Key: CASSANDRA-14103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14103
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>Priority: Minor
> Attachments: 3.11-14103-dtest.png, 3.11-14103-testall.png,
> trunk-14103-dtest.png, trunk-14103-testall.png
>
>
> When the compaction strategies are reloaded after disk boundary changes
> (CASSANDRA-13948), it's possible that a recently finished SSTable is added
> twice to the compaction strategy: once when the compaction strategies are
> reloaded due to the disk boundary change ({{maybeReloadDiskBoundarie}}), and
> another when the {{CompactionStrategyManager}} is processing the
> {{SSTableAddedNotification}}.
> This should be quite unlikely because a compaction must finish as soon as the
> disk boundary changes, and even if it happens most compaction strategies
> would not be affected by it since they deduplicate sstables internally, but
> we should protect against such scenario.
> For more context see [this
> comment|https://issues.apache.org/jira/browse/CASSANDRA-13948?focusedCommentId=16280448=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16280448]
> from Marcus.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14168) Throw error when attempting to mutate non-existant table
[
https://issues.apache.org/jira/browse/CASSANDRA-14168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14168:
Component/s: Distributed Metadata
Core
> Throw error when attempting to mutate non-existant table
>
>
> Key: CASSANDRA-14168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14168
> Project: Cassandra
> Issue Type: Bug
> Components: Core, Distributed Metadata
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>Priority: Major
>
> When a node receives a write request for a table that was just created but
> it's TableMetadata is not yet registered, it will [throw a
> UnknownTableException|https://github.com/apache/cassandra/blob/7a40abb6a5108688fb1b10c375bb751cbb782ea4/src/java/org/apache/cassandra/schema/Schema.java#L474].
> There is a small racy period though between when the {{TableMetadata}} [is
> registered|https://github.com/apache/cassandra/blob/88c0e29caaffab41422adc673182bb9549e735ab/src/java/org/apache/cassandra/schema/Schema.java#L642]
> and it's {{ColumnFamilyStore}} is
> [created|https://github.com/apache/cassandra/blob/88c0e29caaffab41422adc673182bb9549e735ab/src/java/org/apache/cassandra/schema/Schema.java#L643],
> that a write request can be [silently dropped due to ColumnFamilyStore not
> being
> instantiated|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/src/java/org/apache/cassandra/db/Keyspace.java#L603]
> - even though an error is logged, the request will be completed and an ack
> wrongly sent to the coordinator.
> This was detected during investigation of a flakiness on
> {{materialized_views_test.py:TestMaterializedViews.populate_mv_after_insert_wide_rows_test}}.
> ps: this is not an issue before 4.0, because a new table's {{TableMetadata}}
> was only registered after its
> {{ColumnFamilyStore}} object [was
> loaded|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/config/Schema.java#L671]
> (which appears to be changed by CASSANDRA-9425).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14098) Potential Integer Overflow
[
https://issues.apache.org/jira/browse/CASSANDRA-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14098:
Labels: lhf (was: )
> Potential Integer Overflow
> --
>
> Key: CASSANDRA-14098
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14098
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: songwanging
>Priority: Trivial
> Labels: lhf
>
> Our tool DeepTect has detected a potential integer overflow:
> Path: cassandra/src/java/org/apache/cassandra/service/StorageService.java
> {code:java}
> ...
> long totalRowCountEstimate = cfs.estimatedKeysForRange(range);
> ...
> int splitCount = Math.max(1, Math.min(maxSplitCount,
> (int)(totalRowCountEstimate / keysPerSplit)));
> {code}
> In the above code snippet, "totalRowCountEstimate" is a long variable,
> "keysPerSplit" is an integer variable. If "totalRowCountEstimate" is super
> large, directly casting "(totalRowCountEstimate / keysPerSplit" into integer
> will definitely lead to a potential integer overflow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14098) Potential Integer Overflow
[
https://issues.apache.org/jira/browse/CASSANDRA-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14098:
Priority: Trivial (was: Minor)
> Potential Integer Overflow
> --
>
> Key: CASSANDRA-14098
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14098
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: songwanging
>Priority: Trivial
> Labels: lhf
>
> Our tool DeepTect has detected a potential integer overflow:
> Path: cassandra/src/java/org/apache/cassandra/service/StorageService.java
> {code:java}
> ...
> long totalRowCountEstimate = cfs.estimatedKeysForRange(range);
> ...
> int splitCount = Math.max(1, Math.min(maxSplitCount,
> (int)(totalRowCountEstimate / keysPerSplit)));
> {code}
> In the above code snippet, "totalRowCountEstimate" is a long variable,
> "keysPerSplit" is an integer variable. If "totalRowCountEstimate" is super
> large, directly casting "(totalRowCountEstimate / keysPerSplit" into integer
> will definitely lead to a potential integer overflow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14014) Inserting of larger columns leads to exceptions and failing repairs
[
https://issues.apache.org/jira/browse/CASSANDRA-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14014:
Component/s: Core
> Inserting of larger columns leads to exceptions and failing repairs
> ---
>
> Key: CASSANDRA-14014
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14014
> Project: Cassandra
> Issue Type: Bug
> Components: Core, Repair
> Environment: 7 node cluster, debian stretch, 32GB ram, 2TB disk, C*
> 3.11.1
>Reporter: Michael Wohlwend
>Priority: Major
> Attachments: cassandra-log.txt
>
>
> Inserting records with an unusual large text field (hundreds of MB instead of
> a few KB) caused exceptions of the kind "mutation too large":
> ===
> ERROR [MutationStage-3] 2017-11-07 08:32:52,712 StorageProxy.java:1414 -
> Failed to apply mutation locally : {}
> java.lang.IllegalArgumentException: Mutation of 185.439MiB is too large for
> the maximum size of 64.000MiB
> at
> org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:255)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:585)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:462)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:227)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:232)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:241)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at
> org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1408)
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2647)
> [apache-cassandra-3.11.1.jar:3.11.1]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_141]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> [apache-cassandra-3.11.1.jar:3.11.1]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
> [apache-cassandra-3.11.1.jar:3.11.1]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
> [apache-cassandra-3.11.1.jar:3.11.1]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_141]
> WARN [MutationStage-145] 2017-11-07 08:33:08,291
> AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
> Thread[MutationStage-145,5,main]: {}
>
> Trying to repair the table fails. In the end, the cassandra shuts down on the
> node.
> The exceptions thrown during repair are "EOF after xxx bytes out of YYY",
> "leak detected",
> "failed to create merkle tree ... OutOfMemory" and "self-supression not
> permitted".
> See the attached log file (edited and shortened).
> Michael
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14419) Resume compresed hints delivery broken
[
https://issues.apache.org/jira/browse/CASSANDRA-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14419:
Reproduced In: 3.0.16, 3.0.15 (was: 3.0.15, 3.0.16)
Status: Patch Available (was: Open)
> Resume compresed hints delivery broken
> --
>
> Key: CASSANDRA-14419
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14419
> Project: Cassandra
> Issue Type: Bug
> Components: Hints
>Reporter: Tommy Stendahl
>Assignee: Tommy Stendahl
>Priority: Blocker
> Fix For: 3.0.17
>
>
> We are using Cassandra 3.0.15 and are using compressed hints, but if hint
> delivery is interrupted resuming hint delivery is failing.
> {code}
> 2018-04-04T13:27:48.948+0200 ERROR [HintsDispatcher:14]
> CassandraDaemon.java:207 Exception in thread Thread[HintsDispatcher:14,1,main]
> java.lang.IllegalArgumentException: Unable to seek to position 1789149057 in
> /var/lib/cassandra/hints/9592c860-1054-4c60-b3b8-faa9adc6d769-1522838912649-1.hints
> (118259682 bytes) in read-only mode
> at
> org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:287)
> ~[apache-cassandra-clientutil-3.0.15.jar:3.0.15]
> at org.apache.cassandra.hints.HintsReader.seek(HintsReader.java:114)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatcher.seek(HintsDispatcher.java:83)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:263)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:248)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:226)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:205)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_152]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_152]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[na:1.8.0_152]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_152]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [apache-cassandra-3.0.15.jar:3.0.15]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_152]
> {code}
> I think the problem is similar to CASSANDRA-11960.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14419) Resume compresed hints delivery broken
[
https://issues.apache.org/jira/browse/CASSANDRA-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514896#comment-16514896
]
Joshua McKenzie commented on CASSANDRA-14419:
-
"Anonymous" needs to stop setting things to "Ready to Commit" once every two
months. ;)
> Resume compresed hints delivery broken
> --
>
> Key: CASSANDRA-14419
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14419
> Project: Cassandra
> Issue Type: Bug
> Components: Hints
>Reporter: Tommy Stendahl
>Assignee: Tommy Stendahl
>Priority: Blocker
> Fix For: 3.0.17
>
>
> We are using Cassandra 3.0.15 and are using compressed hints, but if hint
> delivery is interrupted resuming hint delivery is failing.
> {code}
> 2018-04-04T13:27:48.948+0200 ERROR [HintsDispatcher:14]
> CassandraDaemon.java:207 Exception in thread Thread[HintsDispatcher:14,1,main]
> java.lang.IllegalArgumentException: Unable to seek to position 1789149057 in
> /var/lib/cassandra/hints/9592c860-1054-4c60-b3b8-faa9adc6d769-1522838912649-1.hints
> (118259682 bytes) in read-only mode
> at
> org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:287)
> ~[apache-cassandra-clientutil-3.0.15.jar:3.0.15]
> at org.apache.cassandra.hints.HintsReader.seek(HintsReader.java:114)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatcher.seek(HintsDispatcher.java:83)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:263)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:248)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:226)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:205)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_152]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_152]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[na:1.8.0_152]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_152]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [apache-cassandra-3.0.15.jar:3.0.15]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_152]
> {code}
> I think the problem is similar to CASSANDRA-11960.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14419) Resume compresed hints delivery broken
[
https://issues.apache.org/jira/browse/CASSANDRA-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14419:
Reproduced In: 3.0.16, 3.0.15 (was: 3.0.15, 3.0.16)
Status: Open (was: Ready to Commit)
> Resume compresed hints delivery broken
> --
>
> Key: CASSANDRA-14419
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14419
> Project: Cassandra
> Issue Type: Bug
> Components: Hints
>Reporter: Tommy Stendahl
>Assignee: Tommy Stendahl
>Priority: Blocker
> Fix For: 3.0.17
>
>
> We are using Cassandra 3.0.15 and are using compressed hints, but if hint
> delivery is interrupted resuming hint delivery is failing.
> {code}
> 2018-04-04T13:27:48.948+0200 ERROR [HintsDispatcher:14]
> CassandraDaemon.java:207 Exception in thread Thread[HintsDispatcher:14,1,main]
> java.lang.IllegalArgumentException: Unable to seek to position 1789149057 in
> /var/lib/cassandra/hints/9592c860-1054-4c60-b3b8-faa9adc6d769-1522838912649-1.hints
> (118259682 bytes) in read-only mode
> at
> org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:287)
> ~[apache-cassandra-clientutil-3.0.15.jar:3.0.15]
> at org.apache.cassandra.hints.HintsReader.seek(HintsReader.java:114)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatcher.seek(HintsDispatcher.java:83)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:263)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:248)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:226)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:205)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_152]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_152]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[na:1.8.0_152]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_152]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [apache-cassandra-3.0.15.jar:3.0.15]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_152]
> {code}
> I think the problem is similar to CASSANDRA-11960.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14518) Want to install cassandra on the single server with mulitple nodes on centos Single server. I will attach the specs of the server
[ https://issues.apache.org/jira/browse/CASSANDRA-14518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie resolved CASSANDRA-14518. - Resolution: Invalid This Jira is for tracking development on the project. Please reference the community site for FAQs and the [user mailing list|https://cassandra.apache.org/community/] for help setting up a cluster. > Want to install cassandra on the single server with mulitple nodes on centos > Single server. I will attach the specs of the server > - > > Key: CASSANDRA-14518 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14518 > Project: Cassandra > Issue Type: Task > Components: Hints > Environment: |Server Specifications|1| > | | | > |Operating System|centos| > |Control Panel|None| > |Additional RAM|64| > |1st and 2nd Hard Disk|2 x 250 GB SSD - RAID 1 - FREE| > |3rd Hard Disk|500 GB SSD| > |4th Hard Disk|500 GB HDD| > |5th Hard Disk|None| > |6th Hard Disk|None| > |Hardware RAID|No| > |Port Speed|100 Mbps - FREE| > |Tier 1 Bandwidth|15 TB - FREE| > |Remote Backup Storage|None| > |IPMI KVM|None| > |Model|79| > |Model_Name|Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz| > |Number of cores|32| >Reporter: girish >Priority: Major > > Datacenter: datacenter1 > === > Status=Up/Down > |/ State=Normal/Leaving/Joining/Moving > -- Address Load Tokens Owns Host ID Rack > UN 127.0.0.1 320.67 KiB 256 ? efb256fb-a04f-4c4f-948b-b105a7f7a658 rack1 > Note: Non-system keyspaces don't have the same replication settings, > effective ownership information is meaningless -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14519) have issues configuring the Cassandra on the
[ https://issues.apache.org/jira/browse/CASSANDRA-14519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie resolved CASSANDRA-14519. - Resolution: Invalid As on the previous ticket, please take usage questions to the mailing list. > have issues configuring the Cassandra on the > - > > Key: CASSANDRA-14519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14519 > Project: Cassandra > Issue Type: Test > Components: Core >Reporter: girish >Priority: Major > > I have installed the Cassandra with multiple nodes but I was not getting any > nodetools status. > where Should I get this done. > > Take a look at this issue: > [root@server2 opt]# nodetool status > Datacenter: datacenter1 > === > Status=Up/Down > |/ State=Normal/Leaving/Joining/Moving > -- Address Load Tokens Owns Host ID Rack > UN 127.0.0.1 320.67 KiB 256 ? efb256fb-a04f-4c4f-948b-b105a7f7 a658 rack1 > Note: Non-system keyspaces don't have the same replication settings, > effective o wnership information is meaningless. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Issue Comment Deleted] (CASSANDRA-14466) Enable Direct I/O
[ https://issues.apache.org/jira/browse/CASSANDRA-14466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14466: Comment: was deleted (was: I am out of the office returning returning Monday May 18. For urgent matters please contact my manager: eric.kaczma...@intel.com Thanks, Tony ) > Enable Direct I/O > -- > > Key: CASSANDRA-14466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14466 > Project: Cassandra > Issue Type: New Feature > Components: Local Write-Read Paths >Reporter: Mulugeta Mammo >Priority: Major > Attachments: direct_io.patch > > > Hi, > JDK 10 introduced a new API for Direct IO that enables applications to bypass > the file system cache and potentially improve performance. Details of this > feature can be found at [https://bugs.openjdk.java.net/browse/JDK-8164900]. > This patch uses the JDK 10 API to enable Direct IO for the Cassandra read > path. By default, we have disabled this feature; but it can be enabled using > a new configuration parameter, enable_direct_io_for_read_path. We have > conducted a Cassandra read-only stress test and measured a throughput gain of > up to 60% on flash drives. > The patch requires JDK 10 Cassandra Support - > https://issues.apache.org/jira/browse/CASSANDRA-9608 > Please review the patch and let us know your feedback. > Thanks, > [^direct_io.patch] > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14464) stop-server.bat -p ../pid.txt -f command not working on windows 2016
[
https://issues.apache.org/jira/browse/CASSANDRA-14464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499077#comment-16499077
]
Joshua McKenzie edited comment on CASSANDRA-14464 at 6/2/18 1:57 PM:
-
When you run with -f, you're getting a simple:
{quote}taskkill /f /pid $pidToKill
{quote}
So seems like there's something odd happening on your system if it's restarting
cassandra after being force-killed by the OS. So first off, I'd recommend not
running with /f since that's basically forcing a hard stop of the DB w/out
letting it shutdown gracefully.
To give some context here on the surprising complexity of C* shutdown on
windows: this code was written in July of 2014, and [does the
following|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L75-L95])
:
# A .bat file calls
# A powershell file which
# Contains a C# program it interprets on the fly
([PowerStopper.Stopper|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L69-L71])
which
# Uses pinvoke to use [native
calls|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L120-L140]
to attach to the console and send a ctrl+C to kill it and let the C*/JVM
signal handlers terminate the application
The TL;DR here: it's kind of a small miracle it worked in the first place
(Windows signal handling very very very much != *nix), and if your two OS
environments are respawning a process that terminated itself due to receiving a
SIGINT and shutting down, there's not much we can do from a runtime script
perspective to stop it.
That being said, I tested the functionality of this on server 2012 back when I
wrote it and it worked fine on a few different environments and used it for dev
purposes daily for a couple years after that on Win10 and Server 2012 without
ever having an issue with it.
The new pid that gets spawned is another java process running C*? If you run
process explorer, what parent process owns that new pid? Is it owned by the
original cassandra.bat that you were running things from? Is it running as a
service under svchost?
was (Author: joshuamckenzie):
When you run with -f, you're getting a simple:
{quote}taskkill /f /pid $pidToKill
{quote}
So seems like there's something odd happening on your system if it's restarting
cassandra after being force-killed by the OS. So first off, I'd recommend not
running with /f since that's basically forcing a hard stop of the DB w/out
letting it shutdown gracefully.
To give some context here on the surprising complexity of C* shutdown on
windows: this code was written in July of 2014, and [does the
following|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L75-L95])
:
# A .bat file calls
# A powershell file which
# Contains a C# program it interprets on the fly
([PowerStopper.Stopper|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L69-L71])
which
# Uses pinvoke to use [native
calls|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L120-L140]
to attach to the console and send a ctrl+C to kill it and let the C*/JVM
signal handlers terminate the application
The TL;DR here: it's kind of a small miracle it worked in the first place
(Windows signal handling very very very much != *nix), and if your two OS
environments are respawning a process that terminated itself due to receiving a
SIGINT and shutting down, there's not much we can do from a runtime script
perspective to stop it.
That being said, I tested the functionality of this on server 2012 back when I
wrote it and it worked fine on a few different environments and used it for dev
purposes daily for a couple years after that on Win10 and Server 2012 for a
couple years after that without ever having an issue with it.
The new pid that gets spawned is another java process running C*? If you run
process explorer, what parent process owns that new pid? Is it owned by the
original cassandra.bat that you were running things from? Is it running as a
service under svchost?
> stop-server.bat -p ../pid.txt -f command not working on windows 2016
>
>
> Key: CASSANDRA-14464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14464
> Project: Cassandra
> Issue Type: Bug
>Reporter: Shyam Phirke
>Priority: Critical
>
> Steps to reproduce:
> 1. Copy and extract cassandra binaries on windows 2016 machine
> 2. Start cassandra in non-legacy mode
> 3. Check pid of cassandra in task manager and compare it with in pid.txt
> 4. Now stop cassandra using command stop-server.bat -p ../pid.txt -f
> Expected:
> After executing \bin:\> stop-server.bat -p
> ../pid.txt -f
> cassandra process as in pid.txt should get killed.
>
>
[jira] [Comment Edited] (CASSANDRA-14464) stop-server.bat -p ../pid.txt -f command not working on windows 2016
[
https://issues.apache.org/jira/browse/CASSANDRA-14464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499077#comment-16499077
]
Joshua McKenzie edited comment on CASSANDRA-14464 at 6/2/18 1:56 PM:
-
When you run with -f, you're getting a simple:
{quote}taskkill /f /pid $pidToKill
{quote}
So seems like there's something odd happening on your system if it's restarting
cassandra after being force-killed by the OS. So first off, I'd recommend not
running with /f since that's basically forcing a hard stop of the DB w/out
letting it shutdown gracefully.
To give some context here on the surprising complexity of C* shutdown on
windows: this code was written in July of 2014, and [does the
following|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L75-L95])
:
# A .bat file calls
# A powershell file which
# Contains a C# program it interprets on the fly
([PowerStopper.Stopper|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L69-L71])
which
# Uses pinvoke to use [native
calls|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L120-L140]
to attach to the console and send a ctrl+C to kill it and let the C*/JVM
signal handlers terminate the application
The TL;DR here: it's kind of a small miracle it worked in the first place
(Windows signal handling very very very much != *nix), and if your two OS
environments are respawning a process that terminated itself due to receiving a
SIGINT and shutting down, there's not much we can do from a runtime script
perspective to stop it.
That being said, I tested the functionality of this on server 2012 back when I
wrote it and it worked fine on a few different environments and used it for dev
purposes daily for a couple years after that on Win10 and Server 2012 for a
couple years after that without ever having an issue with it.
The new pid that gets spawned is another java process running C*? If you run
process explorer, what parent process owns that new pid? Is it owned by the
original cassandra.bat that you were running things from? Is it running as a
service under svchost?
was (Author: joshuamckenzie):
When you run with -f, you're getting a simple:
{quote}taskkill /f /pid $pidToKill
{quote}
So seems like there's something odd happening on your system if it's restarting
cassandra after being force-killed by the OS. So first off, I'd recommend not
running with /f since that's basically forcing a hard stop of the DB w/out
letting it shutdown gracefully.
To give some context here on the surprising complexity of C* shutdown on
windows: this code was written in July of 2014, and [does the
following|[https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L75-L95]])
:
# A .bat file calls
# A powershell file which
# Contains a C# program it interprets on the fly
([PowerStopper.Stopper|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L69-L71])
which
# Uses pinvoke to use [native
calls|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L120-L140]
to attach to the console and send a ctrl+C to kill it and let the C*/JVM
signal handlers terminate the application
The TL;DR here: it's kind of a small miracle it worked in the first place
(Windows signal handling very very very much != *nix), and if your two OS
environments are respawning a process that terminated itself due to receiving a
SIGINT and shutting down, there's not much we can do from a runtime script
perspective to stop it.
That being said, I tested the functionality of this on server 2012 back when I
wrote it and it worked fine on a few different environments and used it for dev
purposes daily for a couple years after that on Win10 and Server 2012 for a
couple years after that without ever having an issue with it.
The new pid that gets spawned is another java process running C*? If you run
process explorer, what parent process owns that new pid? Is it owned by the
original cassandra.bat that you were running things from? Is it running as a
service under svchost?
> stop-server.bat -p ../pid.txt -f command not working on windows 2016
>
>
> Key: CASSANDRA-14464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14464
> Project: Cassandra
> Issue Type: Bug
>Reporter: Shyam Phirke
>Priority: Critical
>
> Steps to reproduce:
> 1. Copy and extract cassandra binaries on windows 2016 machine
> 2. Start cassandra in non-legacy mode
> 3. Check pid of cassandra in task manager and compare it with in pid.txt
> 4. Now stop cassandra using command stop-server.bat -p ../pid.txt -f
> Expected:
> After executing \bin:\> stop-server.bat -p
> ../pid.txt -f
> cassandra process as in
[jira] [Comment Edited] (CASSANDRA-14464) stop-server.bat -p ../pid.txt -f command not working on windows 2016
[
https://issues.apache.org/jira/browse/CASSANDRA-14464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499077#comment-16499077
]
Joshua McKenzie edited comment on CASSANDRA-14464 at 6/2/18 1:55 PM:
-
When you run with -f, you're getting a simple:
{quote}taskkill /f /pid $pidToKill
{quote}
So seems like there's something odd happening on your system if it's restarting
cassandra after being force-killed by the OS. So first off, I'd recommend not
running with /f since that's basically forcing a hard stop of the DB w/out
letting it shutdown gracefully.
To give some context here on the surprising complexity of C* shutdown on
windows: this code was written in July of 2014, and [does the
following|[https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L75-L95]])
:
# A .bat file calls
# A powershell file which
# Contains a C# program it interprets on the fly
([PowerStopper.Stopper|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L69-L71])
which
# Uses pinvoke to use [native
calls|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L120-L140]
to attach to the console and send a ctrl+C to kill it and let the C*/JVM
signal handlers terminate the application
The TL;DR here: it's kind of a small miracle it worked in the first place
(Windows signal handling very very very much != *nix), and if your two OS
environments are respawning a process that terminated itself due to receiving a
SIGINT and shutting down, there's not much we can do from a runtime script
perspective to stop it.
That being said, I tested the functionality of this on server 2012 back when I
wrote it and it worked fine on a few different environments and used it for dev
purposes daily for a couple years after that on Win10 and Server 2012 for a
couple years after that without ever having an issue with it.
The new pid that gets spawned is another java process running C*? If you run
process explorer, what parent process owns that new pid? Is it owned by the
original cassandra.bat that you were running things from? Is it running as a
service under svchost?
was (Author: joshuamckenzie):
When you run with -f, you're getting a simple:
{quote}taskkill /f /pid $pidToKill{quote}
So seems like there's something odd happening on your system if it's restarting
cassandra after being force-killed by the OS. So first off, I'd recommend not
running with /f since that's basically forcing a hard stop of the DB w/out
letting it shutdown gracefully.
To give some context here on the surprising complexity of C* shutdown on
windows: this code was written in July of 2014, and [does the
following|[https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L75-L95]|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L75-L95]:])
:
# A .bat file calls
# A powershell file which
# Contains a C# program it interprets on the fly
([PowerStopper.Stopper|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L69-L71])
which
# Uses pinvoke to use [native
calls|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L120-L140]
to attach to the console and send a ctrl+C to kill it and let the C*/JVM
signal handlers terminate the application
The TL;DR here: it's kind of a small miracle it worked in the first place
(Windows signal handling very very very much != *nix), and if your two OS
environments are respawning a process that terminated itself due to receiving a
SIGINT and shutting down, there's not much we can do from a runtime script
perspective to stop it.
That being said, I tested the functionality of this on server 2012 back when I
wrote it and it worked fine on a few different environments and used it for dev
purposes daily for a couple years after that on Win10 and Server 2012 for a
couple years after that without ever having an issue with it.
The new pid that gets spawned is another java process running C*? If you run
process explorer, what parent process owns that new pid? Is it owned by the
original cassandra.bat that you were running things from? Is it running as a
service under svchost?
> stop-server.bat -p ../pid.txt -f command not working on windows 2016
>
>
> Key: CASSANDRA-14464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14464
> Project: Cassandra
> Issue Type: Bug
>Reporter: Shyam Phirke
>Priority: Critical
>
> Steps to reproduce:
> 1. Copy and extract cassandra binaries on windows 2016 machine
> 2. Start cassandra in non-legacy mode
> 3. Check pid of cassandra in task manager and compare it with in pid.txt
> 4. Now stop cassandra using command stop-server.bat -p ../pid.txt -f
> Expected:
>
[jira] [Commented] (CASSANDRA-14464) stop-server.bat -p ../pid.txt -f command not working on windows 2016
[
https://issues.apache.org/jira/browse/CASSANDRA-14464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499077#comment-16499077
]
Joshua McKenzie commented on CASSANDRA-14464:
-
When you run with -f, you're getting a simple:
{quote}taskkill /f /pid $pidToKill{quote}
So seems like there's something odd happening on your system if it's restarting
cassandra after being force-killed by the OS. So first off, I'd recommend not
running with /f since that's basically forcing a hard stop of the DB w/out
letting it shutdown gracefully.
To give some context here on the surprising complexity of C* shutdown on
windows: this code was written in July of 2014, and [does the
following|[https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L75-L95]|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L75-L95]:])
:
# A .bat file calls
# A powershell file which
# Contains a C# program it interprets on the fly
([PowerStopper.Stopper|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L69-L71])
which
# Uses pinvoke to use [native
calls|https://github.com/apache/cassandra/blob/cassandra-3.11/bin/stop-server.ps1#L120-L140]
to attach to the console and send a ctrl+C to kill it and let the C*/JVM
signal handlers terminate the application
The TL;DR here: it's kind of a small miracle it worked in the first place
(Windows signal handling very very very much != *nix), and if your two OS
environments are respawning a process that terminated itself due to receiving a
SIGINT and shutting down, there's not much we can do from a runtime script
perspective to stop it.
That being said, I tested the functionality of this on server 2012 back when I
wrote it and it worked fine on a few different environments and used it for dev
purposes daily for a couple years after that on Win10 and Server 2012 for a
couple years after that without ever having an issue with it.
The new pid that gets spawned is another java process running C*? If you run
process explorer, what parent process owns that new pid? Is it owned by the
original cassandra.bat that you were running things from? Is it running as a
service under svchost?
> stop-server.bat -p ../pid.txt -f command not working on windows 2016
>
>
> Key: CASSANDRA-14464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14464
> Project: Cassandra
> Issue Type: Bug
>Reporter: Shyam Phirke
>Priority: Critical
>
> Steps to reproduce:
> 1. Copy and extract cassandra binaries on windows 2016 machine
> 2. Start cassandra in non-legacy mode
> 3. Check pid of cassandra in task manager and compare it with in pid.txt
> 4. Now stop cassandra using command stop-server.bat -p ../pid.txt -f
> Expected:
> After executing \bin:\> stop-server.bat -p
> ../pid.txt -f
> cassandra process as in pid.txt should get killed.
>
> Actual:
> After executing above stop command, the cassandra process as in pid.txt gets
> killed but a new process gets created with new pid. Also the pid.txt not
> updated with new pid.
> This new process should not get created.
>
> Please comment on this issue if more details required.
> I am using cassandra 3.11.2.
>
> This issue impacting me much since because of this new process getting
> created my application uninstallation getting impacted.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12924) GraphiteReporter does not reconnect if graphite restarts
[
https://issues.apache.org/jira/browse/CASSANDRA-12924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16468796#comment-16468796
]
Joshua McKenzie commented on CASSANDRA-12924:
-
{quote}presumably contains no breaking API changes
{quote}
+0 from me as well w/this assumption, +1 if it's known. In agreement w/ay on
this one.
> GraphiteReporter does not reconnect if graphite restarts
>
>
> Key: CASSANDRA-12924
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12924
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stefano Ortolani
>Assignee: mck
>Priority: Major
> Fix For: 3.11.3
>
>
> Seems like GraphiteReporter does not reconnect after graphite is restarted.
> The consequence is complete loss of reported metrics until Cassandra
> restarts. Logs show this every minute:
> {noformat}
> WARN [metrics-graphite-reporter-1-thread-1] 2016-11-17 10:06:26,549
> GraphiteReporter.java:179 - Unable to report to Graphite
> java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> ~[na:1.8.0_91]
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
> ~[na:1.8.0_91]
> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> ~[na:1.8.0_91]
> at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
> ~[na:1.8.0_91]
> at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
> ~[na:1.8.0_91]
> at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[na:1.8.0_91]
> at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> ~[na:1.8.0_91]
> at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> ~[na:1.8.0_91]
> at java.io.BufferedWriter.write(BufferedWriter.java:230) ~[na:1.8.0_91]
> at java.io.Writer.write(Writer.java:157) ~[na:1.8.0_91]
> at com.codahale.metrics.graphite.Graphite.send(Graphite.java:130)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:283)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162)
> [metrics-core-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117)
> [metrics-core-3.1.0.jar:3.1.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_91]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_91]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_91]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_91]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> WARN [metrics-graphite-reporter-1-thread-1] 2016-11-17 10:06:26,549
> GraphiteReporter.java:183 - Error closing Graphite
> java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> ~[na:1.8.0_91]
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
> ~[na:1.8.0_91]
> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> ~[na:1.8.0_91]
> at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
> ~[na:1.8.0_91]
> at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
> ~[na:1.8.0_91]
> at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[na:1.8.0_91]
> at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> ~[na:1.8.0_91]
> at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> ~[na:1.8.0_91]
> at java.io.BufferedWriter.write(BufferedWriter.java:230) ~[na:1.8.0_91]
> at java.io.Writer.write(Writer.java:157) ~[na:1.8.0_91]
> at com.codahale.metrics.graphite.Graphite.send(Graphite.java:130)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:283)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162)
> [metrics-core-3.1.0.jar:3.1.0]
>
[jira] [Commented] (CASSANDRA-8743) NFS doesn't behave on Windows
[ https://issues.apache.org/jira/browse/CASSANDRA-8743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454794#comment-16454794 ] Joshua McKenzie commented on CASSANDRA-8743: Most of the issue on the Windows side was that, if anyone anywhere has a file handle open to *anything* linked to the original root file handle (i.e. any other hard links, etc), none of them are allowed to delete any of them. Not cool, and no way around it, just how NTFS rolls. So files over a network share probably had some kernel hook accessing / caching / who knows what (I wouldn't put it past the [Cache Manager|https://docs.microsoft.com/en-us/windows-server/administration/performance-tuning/subsystem/cache-memory-management/] to be up to something) on the remote side, so boom. As such, I didn't move on it any more because a) we recommended not using network shares for data file storage anyway, and b) my digging kept hitting hard places that I couldn't work around. Somewhat surprised you see that behavior on Linux. Linux tends to move handles to the /proc/$pid subsystem when things are deleted but a process still has a ref to the handle open somewhere, but I'm not sure what/how NFS does that on the remote end when asked to delete something. Your hypothesis doesn't seem that out of line to me, but yeah - parallel repair would be the immediate workaround for this. > NFS doesn't behave on Windows > - > > Key: CASSANDRA-8743 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8743 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths, Streaming and Messaging >Reporter: Tamar Nirenberg >Priority: Minor > Fix For: 2.2.x > > Attachments: docker-system.log > > > Running repair over NFS in Cassandra 2.1.2 encounters this error and crashes > the ring: > ERROR [ValidationExecutor:2] 2015-01-22 11:48:14,811 Validator.java:232 - > Failed creating a merkle tree for [repair > #c84c7c70-a21b-11e4-aeca-19e6d7fa2595 on ATTRIBUTES/LINKS, > (11621838520493020277529637175352775759,11853478749048239324667887059881170862]], > /10.1.234.63 (see log for details) > ERROR [ValidationExecutor:2] 2015-01-22 11:48:14,827 CassandraDaemon.java:153 > - Exception in thread Thread[ValidationExecutor:2,1,main] > org.apache.cassandra.io.FSWriteError: > java.nio.file.DirectoryNotEmptyException: > /exlibris/cassandra/local/data/data/ATTRIBUTES/LINKS/snapshots/c84c7c70-a21b-11e4-aeca-19e6d7fa2595 > at > org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:135) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at > org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:381) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at > org.apache.cassandra.db.Directories.clearSnapshot(Directories.java:547) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at > org.apache.cassandra.db.ColumnFamilyStore.clearSnapshot(ColumnFamilyStore.java:2223) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at > org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:939) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at > org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:97) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at > org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:557) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > ~[na:1.7.0_71] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > ~[na:1.7.0_71] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_71] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71] > Caused by: java.nio.file.DirectoryNotEmptyException: > /exlibris/cassandra/local/data/data/ATTRIBUTES/LINKS/snapshots/c84c7c70-a21b-11e4-aeca-19e6d7fa2595 > at > sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:242) > ~[na:1.7.0_71] > at > sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) > ~[na:1.7.0_71] > at java.nio.file.Files.delete(Files.java:1079) ~[na:1.7.0_71] > at > org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:131) > ~[apache-cassandra-2.1.2.jar:2.1.2] > ... 10 common frames omitted > ERROR [ValidationExecutor:2] 2015-01-22 11:48:14,829 StorageService.java:383 > - Stopping gossiper > WARN [ValidationExecutor:2] 2015-01-22 11:48:14,829 StorageService.java:291 > - Stopping gossip by operator request > INFO [ValidationExecutor:2] 2015-01-22 11:48:14,829 Gossiper.java:1318 - > Announcing shutdown -- This message was sent by Atlassian JIRA
[jira] [Commented] (CASSANDRA-14418) Cassandra not getting started when using enhanced startup scripts in windows
[ https://issues.apache.org/jira/browse/CASSANDRA-14418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454780#comment-16454780 ] Joshua McKenzie commented on CASSANDRA-14418: - This fix immediately passes the smell test for me. Those double quotes on env vars were the bane of my existence when I was working on those scripts. I'll leave you to the testing Dinesh, but I have high hopes. > Cassandra not getting started when using enhanced startup scripts in windows > > > Key: CASSANDRA-14418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14418 > Project: Cassandra > Issue Type: Bug >Reporter: Shyam Phirke >Priority: Major > Fix For: 3.11.x > > > I am using Apache Cassandra 3.11.2 with my application. > My application is getting installed under C:/Program Files/My > Application/Some Folder/. > And cassandra C:/Program Files/My Application/Some Folder/cassandra. > So when I am using enhanced startup scripts cassandra not getting up and > running and I am getting below error: > "Error: Could not find or load main class Files\My" > One of the solution I got is moving cassandra to another location where > location path does not contain spaces. But this is not good way of getting > this problem resolved. > After doing detailed analysis of all the scripts I found the solution below: > Inside file cassandra-env.ps1 at line number 380: > Replace line: > $env:JVM_OPTS = "$env:JVM_OPTS > -XX:CompileCommandFile=$env:CASSANDRA_CONF\hotspot_compiler" > with line > $env:JVM_OPTS = "$env:JVM_OPTS > -XX:CompileCommandFile=""$env:CASSANDRA_CONF\hotspot_compiler""" > Fix here is the double quotes added before $env:CASSANDRA_CONF and at the end. > At other places this case is well handled. But missed at this place. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14349) Untracked CDC segment files are not deleted after replay
[ https://issues.apache.org/jira/browse/CASSANDRA-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430554#comment-16430554 ] Joshua McKenzie commented on CASSANDRA-14349: - Yeah, didn't think about it being the whole "push it, github PR's don't work" thing. I'll get to it in a bit. > Untracked CDC segment files are not deleted after replay > > > Key: CASSANDRA-14349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14349 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > When CDC is enabled, a hard link to each commit log file will be created in > cdc_raw directory. Those commit logs with CDC mutations will also have cdc > index files created along with the hard links; these are intended for the > consumer to handle and clean them up. > However, if we don't produce any CDC traffic, those hard links in cdc_raw > will be never cleaned up (because hard links will still be created, without > the index files), whereas the real original commit logs are correctly deleted > after replay during process startup. This will results in many untracked hard > links in cdc_raw if we restart the cassandra process many times. I am able to > use CCM to reproduce it in trunk version which has the CASSANDRA-12148 > changes. > This seems a bug in handleReplayedSegment of the commit log segment manager > which neglects to take care of CDC commit logs. I will attach a patch here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14349) Untracked CDC segment files are not deleted after replay
[ https://issues.apache.org/jira/browse/CASSANDRA-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429789#comment-16429789 ] Joshua McKenzie commented on CASSANDRA-14349: - Merged C* side. Don't have merge rights on the dtest repo, so pinged [~philipthompson] about that. > Untracked CDC segment files are not deleted after replay > > > Key: CASSANDRA-14349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14349 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > When CDC is enabled, a hard link to each commit log file will be created in > cdc_raw directory. Those commit logs with CDC mutations will also have cdc > index files created along with the hard links; these are intended for the > consumer to handle and clean them up. > However, if we don't produce any CDC traffic, those hard links in cdc_raw > will be never cleaned up (because hard links will still be created, without > the index files), whereas the real original commit logs are correctly deleted > after replay during process startup. This will results in many untracked hard > links in cdc_raw if we restart the cassandra process many times. I am able to > use CCM to reproduce it in trunk version which has the CASSANDRA-12148 > changes. > This seems a bug in handleReplayedSegment of the commit log segment manager > which neglects to take care of CDC commit logs. I will attach a patch here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14349) Untracked CDC segment files are not deleted after replay
[ https://issues.apache.org/jira/browse/CASSANDRA-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14349: Status: Ready to Commit (was: Patch Available) > Untracked CDC segment files are not deleted after replay > > > Key: CASSANDRA-14349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14349 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > When CDC is enabled, a hard link to each commit log file will be created in > cdc_raw directory. Those commit logs with CDC mutations will also have cdc > index files created along with the hard links; these are intended for the > consumer to handle and clean them up. > However, if we don't produce any CDC traffic, those hard links in cdc_raw > will be never cleaned up (because hard links will still be created, without > the index files), whereas the real original commit logs are correctly deleted > after replay during process startup. This will results in many untracked hard > links in cdc_raw if we restart the cassandra process many times. I am able to > use CCM to reproduce it in trunk version which has the CASSANDRA-12148 > changes. > This seems a bug in handleReplayedSegment of the commit log segment manager > which neglects to take care of CDC commit logs. I will attach a patch here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14349) Untracked CDC segment files are not deleted after replay
[ https://issues.apache.org/jira/browse/CASSANDRA-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429555#comment-16429555 ] Joshua McKenzie commented on CASSANDRA-14349: - +1 to both patches; I'll get the C* side committed tomorrow. Open a PR for those dtest changes and I'll get those merged after the C* side. Thanks! > Untracked CDC segment files are not deleted after replay > > > Key: CASSANDRA-14349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14349 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > When CDC is enabled, a hard link to each commit log file will be created in > cdc_raw directory. Those commit logs with CDC mutations will also have cdc > index files created along with the hard links; these are intended for the > consumer to handle and clean them up. > However, if we don't produce any CDC traffic, those hard links in cdc_raw > will be never cleaned up (because hard links will still be created, without > the index files), whereas the real original commit logs are correctly deleted > after replay during process startup. This will results in many untracked hard > links in cdc_raw if we restart the cassandra process many times. I am able to > use CCM to reproduce it in trunk version which has the CASSANDRA-12148 > changes. > This seems a bug in handleReplayedSegment of the commit log segment manager > which neglects to take care of CDC commit logs. I will attach a patch here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Issue Comment Deleted] (CASSANDRA-13896) Improving Cassandra write performance
[ https://issues.apache.org/jira/browse/CASSANDRA-13896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-13896: Comment: was deleted (was: I am out of the office returning Tuesday April 3. For urgent matters please contact my manager: eric.kaczma...@intel.com Thanks, Tony ) > Improving Cassandra write performance > --- > > Key: CASSANDRA-13896 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13896 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths > Environment: Skylake server with 2 sockets, 192GB RAM, 3xPCIe SSDs > OS: Centos 7.3 > Java: Oracle JDK1.8.0_121 >Reporter: Prince Nana Owusu Boateng >Priority: Major > Labels: Performance > Fix For: 4.x > > Attachments: Screen Shot 2017-09-22 at 11.22.43 AM.png, Screen Shot > 2017-09-22 at 3.31.09 PM.png > > > During our Cassandra performance testing, we see high percentage of the CPU > spent in *org.apache.cassandra.utils.memory.SlabAllocator.allocate(int, > OpOrder Group) * method. Appears to be high contention of the > ** atomic Integer in write workloads. This structure is > used by the threads for keeping track of the region bytebuffer allocation. > When the contention appears, adding more clients, modifications of write > specific parameters does not change write throughput performance. Attached > are the details of Java Flight Recorder (JFR), showing hot functions and also > performance results. When we see this contention, we still have plenty of > CPU and throughput left ( *<20%* Total average CPU utilization and *<11%* > of the storage write total throughput). This occurs on Cassandra 3.10.0-src > version using the Cassandra-Stress. > Proposal: > We will like to introduce a solution which eliminates the atomic operations > on the ** atomic Integer. This implementation will allow > concurrent allocation of bytebuffers without an atomic compareAndSet and > incrementAndGet operations. The solution is expected to increase overall > write performance while improving CPU utilization. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14349) Untracked CDC segment files are not deleted after replay
[ https://issues.apache.org/jira/browse/CASSANDRA-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422621#comment-16422621 ] Joshua McKenzie commented on CASSANDRA-14349: - I'll take review on it. Should be able to get to it this week. > Untracked CDC segment files are not deleted after replay > > > Key: CASSANDRA-14349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14349 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > When CDC is enabled, a hard link to each commit log file will be created in > cdc_raw directory. Those commit logs with CDC mutations will also have cdc > index files created along with the hard links; these are intended for the > consumer to handle and clean them up. > However, if we don't produce any CDC traffic, those hard links in cdc_raw > will be never cleaned up (because hard links will still be created, without > the index files), whereas the real original commit logs are correctly deleted > after replay during process startup. This will results in many untracked hard > links in cdc_raw if we restart the cassandra process many times. I am able to > use CCM to reproduce it in trunk version which has the CASSANDRA-12148 > changes. > This seems a bug in handleReplayedSegment of the commit log segment manager > which neglects to take care of CDC commit logs. I will attach a patch here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14349) Untracked CDC segment files are not deleted after replay
[ https://issues.apache.org/jira/browse/CASSANDRA-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14349: Reviewer: Joshua McKenzie > Untracked CDC segment files are not deleted after replay > > > Key: CASSANDRA-14349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14349 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > When CDC is enabled, a hard link to each commit log file will be created in > cdc_raw directory. Those commit logs with CDC mutations will also have cdc > index files created along with the hard links; these are intended for the > consumer to handle and clean them up. > However, if we don't produce any CDC traffic, those hard links in cdc_raw > will be never cleaned up (because hard links will still be created, without > the index files), whereas the real original commit logs are correctly deleted > after replay during process startup. This will results in many untracked hard > links in cdc_raw if we restart the cassandra process many times. I am able to > use CCM to reproduce it in trunk version which has the CASSANDRA-12148 > changes. > This seems a bug in handleReplayedSegment of the commit log segment manager > which neglects to take care of CDC commit logs. I will attach a patch here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14339) Modernize Internal Vocabulary
[ https://issues.apache.org/jira/browse/CASSANDRA-14339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412741#comment-16412741 ] Joshua McKenzie commented on CASSANDRA-14339: - bq. and "wide row" = partition Thinking leaving partition alone would be best. > Modernize Internal Vocabulary > - > > Key: CASSANDRA-14339 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14339 > Project: Cassandra > Issue Type: Task > Components: Core >Reporter: Jon Haddad >Assignee: Jon Haddad >Priority: Major > Fix For: 4.x > > > This is an umbrella ticket for all modifications necessary to update the code > base from old Thrift-centric terminology to CQL terminology. References to > Column families should be completely removed at the close of this issue, rows > should mean CQL rows, and "wide row" = partition. > Each sub ticket should be as small of a refactor as possible. For instance, > the rename of a single class should only touch the class, with a follow up > ticket for the variable names, methods, etc, to keep the scope of each > refactor to a minimum. The exception to this are the comments that are > associated with a block of code, which should be addressed with the ticket. > Ensure changes that affect JMX are thoroughly tested, this should be covered > by our utests & dtests, but care should be taken in case they are not. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14118) Refactor write path
[ https://issues.apache.org/jira/browse/CASSANDRA-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404752#comment-16404752 ] Joshua McKenzie edited comment on CASSANDRA-14118 at 3/19/18 12:45 PM: --- bq. And yes, not using the C* commit log would probably preclude supporting cdc. Correct. Its current implementation is constrained by the 1:1 requirement (if it persists to disk, it's available for consumption, if it doesn't, it's not) + desire to keep heap pressure from the feature to a minimum (i.e. don't cache AllTheThings, then mark them in memory as persisted on CL flush, then fragment that memory out if offheap or pressure GC if on on reclaim, etc etc etc). i.e. it's using the CL as the mechanism of persistence + "communication" with an external process reading the files, so you don't write it to the CL, you don't get "CDC". was (Author: joshuamckenzie): bq. And yes, not using the C* commit log would probably preclude supporting cdc. Correct. It's current implementation is constrained by the 1:1 requirement (if it persists to disk, it's available for consumption, if it doesn't, it's not) + desire to keep heap pressure from the feature to a minimum (i.e. don't cache AllTheThings, then mark them in memory as persisted on CL flush, then fragment that memory out if offheap or pressure GC if on on reclaim, etc etc etc). i.e. it's using the CL as the mechanism of persistence + "communication" with an external process reading the files, so you don't write it to the CL, you don't get "CDC". > Refactor write path > --- > > Key: CASSANDRA-14118 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14118 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Dikang Gu >Assignee: Dikang Gu >Priority: Major > > As part of the pluggable storage engine effort, we'd like to modularize the > write path related code, make it to be independent from existing storage > engine implementation details. > For now, refer to > https://docs.google.com/document/d/1suZlvhzgB6NIyBNpM9nxoHxz_Ri7qAm-UEO8v8AIFsc > for high level designs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14118) Refactor write path
[ https://issues.apache.org/jira/browse/CASSANDRA-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404752#comment-16404752 ] Joshua McKenzie commented on CASSANDRA-14118: - bq. And yes, not using the C* commit log would probably preclude supporting cdc. Correct. It's current implementation is constrained by the 1:1 requirement (if it persists to disk, it's available for consumption, if it doesn't, it's not) + desire to keep heap pressure from the feature to a minimum (i.e. don't cache AllTheThings, then mark them in memory as persisted on CL flush, then fragment that memory out if offheap or pressure GC if on on reclaim, etc etc etc). i.e. it's using the CL as the mechanism of persistence + "communication" with an external process reading the files, so you don't write it to the CL, you don't get "CDC". > Refactor write path > --- > > Key: CASSANDRA-14118 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14118 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Dikang Gu >Assignee: Dikang Gu >Priority: Major > > As part of the pluggable storage engine effort, we'd like to modularize the > write path related code, make it to be independent from existing storage > engine implementation details. > For now, refer to > https://docs.google.com/document/d/1suZlvhzgB6NIyBNpM9nxoHxz_Ri7qAm-UEO8v8AIFsc > for high level designs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14281) Improve LatencyMetrics performance by reducing write path processing
[ https://issues.apache.org/jira/browse/CASSANDRA-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395183#comment-16395183 ] Joshua McKenzie commented on CASSANDRA-14281: - Bounced out of Ready to Commit but can't move back to PA. Just a heads up. > Improve LatencyMetrics performance by reducing write path processing > > > Key: CASSANDRA-14281 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14281 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Michael Burman >Assignee: Michael Burman >Priority: Major > > Currently for each write/read/rangequery/CAS touching the CFS we write a > latency metric which takes a lot of processing time (up to 66% of the total > processing time if the update was empty). > The way latencies are recorded is to use both a dropwizard "Timer" as well as > "Counter". Latter is used for totalLatency and the previous is decaying > metric for rates and certain percentile metrics. We then replicate all of > these CFS writes to the KeyspaceMetrics and globalWriteLatencies. > Instead of doing this on the write phase we should merge the metrics when > they're read. This is much less common occurrence and thus we save a lot of > CPU time in total. This also speeds up the write path. > Currently, the DecayingEstimatedHistogramReservoir acquires a lock for each > update operation, which causes a contention if there are more than one thread > updating the histogram. This impacts scalability when using larger machines. > We should make it lock-free as much as possible and also avoid a single > CAS-update from blocking all the concurrent threads from making an update. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14281) Improve LatencyMetrics performance by reducing write path processing
[ https://issues.apache.org/jira/browse/CASSANDRA-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14281: Status: In Progress (was: Ready to Commit) > Improve LatencyMetrics performance by reducing write path processing > > > Key: CASSANDRA-14281 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14281 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Michael Burman >Assignee: Michael Burman >Priority: Major > > Currently for each write/read/rangequery/CAS touching the CFS we write a > latency metric which takes a lot of processing time (up to 66% of the total > processing time if the update was empty). > The way latencies are recorded is to use both a dropwizard "Timer" as well as > "Counter". Latter is used for totalLatency and the previous is decaying > metric for rates and certain percentile metrics. We then replicate all of > these CFS writes to the KeyspaceMetrics and globalWriteLatencies. > Instead of doing this on the write phase we should merge the metrics when > they're read. This is much less common occurrence and thus we save a lot of > CPU time in total. This also speeds up the write path. > Currently, the DecayingEstimatedHistogramReservoir acquires a lock for each > update operation, which causes a contention if there are more than one thread > updating the histogram. This impacts scalability when using larger machines. > We should make it lock-free as much as possible and also avoid a single > CAS-update from blocking all the concurrent threads from making an update. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14290) When is 4.0 release planned
[ https://issues.apache.org/jira/browse/CASSANDRA-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie resolved CASSANDRA-14290. - Resolution: Invalid Please take questions like this to the user mailing list. JIRA is for tracking development effort on the project. > When is 4.0 release planned > --- > > Key: CASSANDRA-14290 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14290 > Project: Cassandra > Issue Type: Task >Reporter: Maitrayee >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14261) Compaction Profiling Improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377386#comment-16377386 ] Joshua McKenzie commented on CASSANDRA-14261: - bq. but these looked simple and sufficiently isolated not to warrant a full run in perf. Right. Was thinking more something along the lines of jmh microbenchmarks or something that might be usable in the future as well. What you have here is pretty non-controversial, but performance optimization code-changes w/out perf testing is something we have a pretty long history on this project of having bad hygiene around. > Compaction Profiling Improvements > - > > Key: CASSANDRA-14261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14261 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Minor > Fix For: 4.x > > > There's some low hanging fruit in some laptop compaction runs, such as > creating a ton of the same object unnecessarily and hashing cell names > repeatedly to see if a column is dropped even when we should know that the > table has no dropped columns. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14261) Compaction Profiling Improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376897#comment-16376897 ] Joshua McKenzie commented on CASSANDRA-14261: - You have some perf runs to go with those correctness runs to indicate the impact of the change? > Compaction Profiling Improvements > - > > Key: CASSANDRA-14261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14261 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Minor > Fix For: 4.x > > > There's some low hanging fruit in some laptop compaction runs, such as > creating a ton of the same object unnecessarily and hashing cell names > repeatedly to see if a column is dropped even when we should know that the > table has no dropped columns. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13993) Add optional startup delay to wait until peers are ready
[
https://issues.apache.org/jira/browse/CASSANDRA-13993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358383#comment-16358383
]
Joshua McKenzie commented on CASSANDRA-13993:
-
{quote}wdyt?{quote}
Passes the smell test. Legacy code is such a delight.
Anyone that's relying on extending these verbs can do the leg work to better
integrate with 13283's impl after this change if they haven't yet anyway, as
it's a cleaner solution to this than just "keep adding a little breathing room
so we maybe don't overflow".
> Add optional startup delay to wait until peers are ready
>
>
> Key: CASSANDRA-13993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13993
> Project: Cassandra
> Issue Type: Improvement
> Components: Lifecycle
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Minor
> Fix For: 4.x
>
>
> When bouncing a node in a large cluster, is can take a while to recognize the
> rest of the cluster as available. This is especially true if using TLS on
> internode messaging connections. The bouncing node (and any clients connected
> to it) may see a series of Unavailable or Timeout exceptions until the node
> is 'warmed up' as connecting to the rest of the cluster is asynchronous from
> the rest of the startup process.
> There are two aspects that drive a node's ability to successfully communicate
> with a peer after a bounce:
> - marking the peer as 'alive' (state that is held in gossip). This affects
> the unavailable exceptions
> - having both open outbound and inbound connections open and ready to each
> peer. This affects timeouts.
> Details of each of these mechanisms are described in the comments below.
> This ticket proposes adding a mechanism, optional and configurable, to delay
> opening the client native protocol port until some percentage of the peers in
> the cluster is marked alive and connected to/from. Thus while we potentially
> slow down startup (delay opening the client port), we alleviate the chance
> that queries made by clients don't hit transient unavailable/timeout
> exceptions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13993) Add optional startup delay to wait until peers are ready
[
https://issues.apache.org/jira/browse/CASSANDRA-13993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357409#comment-16357409
]
Joshua McKenzie commented on CASSANDRA-13993:
-
Regardless of whether the unused slots are currently used by other consumers
(be it DSE or otherwise), inserting an enum in the middle explicitly violates
the contract / comment in the code:
{code:java}
// remember to add new verbs at the end, since we serialize by ordinal
UNUSED_1,
UNUSED_2,
UNUSED_3,
UNUSED_4,
UNUSED_5,
;{code}
So I'd recommend against inserting a new verb in the middle.
> Add optional startup delay to wait until peers are ready
>
>
> Key: CASSANDRA-13993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13993
> Project: Cassandra
> Issue Type: Improvement
> Components: Lifecycle
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Minor
> Fix For: 4.x
>
>
> When bouncing a node in a large cluster, is can take a while to recognize the
> rest of the cluster as available. This is especially true if using TLS on
> internode messaging connections. The bouncing node (and any clients connected
> to it) may see a series of Unavailable or Timeout exceptions until the node
> is 'warmed up' as connecting to the rest of the cluster is asynchronous from
> the rest of the startup process.
> There are two aspects that drive a node's ability to successfully communicate
> with a peer after a bounce:
> - marking the peer as 'alive' (state that is held in gossip). This affects
> the unavailable exceptions
> - having both open outbound and inbound connections open and ready to each
> peer. This affects timeouts.
> Details of each of these mechanisms are described in the comments below.
> This ticket proposes adding a mechanism, optional and configurable, to delay
> opening the client native protocol port until some percentage of the peers in
> the cluster is marked alive and connected to/from. Thus while we potentially
> slow down startup (delay opening the client port), we alleviate the chance
> that queries made by clients don't hit transient unavailable/timeout
> exceptions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14195) CommitLogSegmentManagerCDCTest is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-14195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14195:
Resolution: Fixed
Fix Version/s: 4.0
Status: Resolved (was: Ready to Commit)
[Committed|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=69db2359ee0889cb4a57aec179b9821ff442d26b].
Thanks Jay!
> CommitLogSegmentManagerCDCTest is flaky
> ---
>
> Key: CASSANDRA-14195
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14195
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Ariel Weisberg
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 4.0
>
>
> This fails fairly reliably in CircleCI and in a few minutes if you run it in
> a loop on a MacOS laptop.
> I see two failures.
> {noformat}
> [junit] Testcase:
> testRetainLinkOnDiscardCDC(org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest):
>Caused an ERROR
> [junit] Rejecting mutation to keyspace cql_test_keyspace. Free up space
> in build/test/cassandra/cdc_raw:0 by processing CDC logs.
> [junit] org.apache.cassandra.exceptions.CDCWriteException: Rejecting
> mutation to keyspace cql_test_keyspace. Free up space in
> build/test/cassandra/cdc_raw:0 by processing CDC logs.
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC.throwIfForbidden(CommitLogSegmentManagerCDC.java:136)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC.allocate(CommitLogSegmentManagerCDC.java:108)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:272)
> [junit] at
> org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:604)
> [junit] at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:481)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:191)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:196)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:205)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testRetainLinkOnDiscardCDC(CommitLogSegmentManagerCDCTest.java:256)
> {noformat}
> and
> {noformat}
> [junit] Testcase:
> testCompletedFlag(org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest):
> FAILED
> [junit] Index file not written:
> build/test/cassandra/cdc_raw:0/CommitLog-7-1517005121474_cdc.idx
> [junit] junit.framework.AssertionFailedError: Index file not written:
> build/test/cassandra/cdc_raw:0/CommitLog-7-1517005121474_cdc.idx
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testCompletedFlag(CommitLogSegmentManagerCDCTest.java:210)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14195) CommitLogSegmentManagerCDCTest is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-14195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14195:
Status: Ready to Commit (was: Patch Available)
> CommitLogSegmentManagerCDCTest is flaky
> ---
>
> Key: CASSANDRA-14195
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14195
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Ariel Weisberg
>Assignee: Jay Zhuang
>Priority: Minor
>
> This fails fairly reliably in CircleCI and in a few minutes if you run it in
> a loop on a MacOS laptop.
> I see two failures.
> {noformat}
> [junit] Testcase:
> testRetainLinkOnDiscardCDC(org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest):
>Caused an ERROR
> [junit] Rejecting mutation to keyspace cql_test_keyspace. Free up space
> in build/test/cassandra/cdc_raw:0 by processing CDC logs.
> [junit] org.apache.cassandra.exceptions.CDCWriteException: Rejecting
> mutation to keyspace cql_test_keyspace. Free up space in
> build/test/cassandra/cdc_raw:0 by processing CDC logs.
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC.throwIfForbidden(CommitLogSegmentManagerCDC.java:136)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC.allocate(CommitLogSegmentManagerCDC.java:108)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:272)
> [junit] at
> org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:604)
> [junit] at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:481)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:191)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:196)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:205)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testRetainLinkOnDiscardCDC(CommitLogSegmentManagerCDCTest.java:256)
> {noformat}
> and
> {noformat}
> [junit] Testcase:
> testCompletedFlag(org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest):
> FAILED
> [junit] Index file not written:
> build/test/cassandra/cdc_raw:0/CommitLog-7-1517005121474_cdc.idx
> [junit] junit.framework.AssertionFailedError: Index file not written:
> build/test/cassandra/cdc_raw:0/CommitLog-7-1517005121474_cdc.idx
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testCompletedFlag(CommitLogSegmentManagerCDCTest.java:210)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14195) CommitLogSegmentManagerCDCTest is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-14195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345079#comment-16345079
]
Joshua McKenzie commented on CASSANDRA-14195:
-
Not yet; env is super rusty. I should be able to get to it today.
Thankfully it's in a bit of the code that's not changing frequently so I can
get away with being slow. =/
Sorry about that [~jay.zhuang]!
> CommitLogSegmentManagerCDCTest is flaky
> ---
>
> Key: CASSANDRA-14195
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14195
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Ariel Weisberg
>Assignee: Jay Zhuang
>Priority: Minor
>
> This fails fairly reliably in CircleCI and in a few minutes if you run it in
> a loop on a MacOS laptop.
> I see two failures.
> {noformat}
> [junit] Testcase:
> testRetainLinkOnDiscardCDC(org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest):
>Caused an ERROR
> [junit] Rejecting mutation to keyspace cql_test_keyspace. Free up space
> in build/test/cassandra/cdc_raw:0 by processing CDC logs.
> [junit] org.apache.cassandra.exceptions.CDCWriteException: Rejecting
> mutation to keyspace cql_test_keyspace. Free up space in
> build/test/cassandra/cdc_raw:0 by processing CDC logs.
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC.throwIfForbidden(CommitLogSegmentManagerCDC.java:136)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC.allocate(CommitLogSegmentManagerCDC.java:108)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:272)
> [junit] at
> org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:604)
> [junit] at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:481)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:191)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:196)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:205)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testRetainLinkOnDiscardCDC(CommitLogSegmentManagerCDCTest.java:256)
> {noformat}
> and
> {noformat}
> [junit] Testcase:
> testCompletedFlag(org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest):
> FAILED
> [junit] Index file not written:
> build/test/cassandra/cdc_raw:0/CommitLog-7-1517005121474_cdc.idx
> [junit] junit.framework.AssertionFailedError: Index file not written:
> build/test/cassandra/cdc_raw:0/CommitLog-7-1517005121474_cdc.idx
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testCompletedFlag(CommitLogSegmentManagerCDCTest.java:210)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14195) CommitLogSegmentManagerCDCTest is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-14195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343655#comment-16343655
]
Joshua McKenzie commented on CASSANDRA-14195:
-
+1. Good catch.
> CommitLogSegmentManagerCDCTest is flaky
> ---
>
> Key: CASSANDRA-14195
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14195
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Ariel Weisberg
>Assignee: Jay Zhuang
>Priority: Minor
>
> This fails fairly reliably in CircleCI and in a few minutes if you run it in
> a loop on a MacOS laptop.
> I see two failures.
> {noformat}
> [junit] Testcase:
> testRetainLinkOnDiscardCDC(org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest):
>Caused an ERROR
> [junit] Rejecting mutation to keyspace cql_test_keyspace. Free up space
> in build/test/cassandra/cdc_raw:0 by processing CDC logs.
> [junit] org.apache.cassandra.exceptions.CDCWriteException: Rejecting
> mutation to keyspace cql_test_keyspace. Free up space in
> build/test/cassandra/cdc_raw:0 by processing CDC logs.
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC.throwIfForbidden(CommitLogSegmentManagerCDC.java:136)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC.allocate(CommitLogSegmentManagerCDC.java:108)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:272)
> [junit] at
> org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:604)
> [junit] at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:481)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:191)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:196)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:205)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testRetainLinkOnDiscardCDC(CommitLogSegmentManagerCDCTest.java:256)
> {noformat}
> and
> {noformat}
> [junit] Testcase:
> testCompletedFlag(org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest):
> FAILED
> [junit] Index file not written:
> build/test/cassandra/cdc_raw:0/CommitLog-7-1517005121474_cdc.idx
> [junit] junit.framework.AssertionFailedError: Index file not written:
> build/test/cassandra/cdc_raw:0/CommitLog-7-1517005121474_cdc.idx
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testCompletedFlag(CommitLogSegmentManagerCDCTest.java:210)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14195) CommitLogSegmentManagerCDCTest is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-14195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-14195:
Priority: Minor (was: Major)
Reviewer: Joshua McKenzie
> CommitLogSegmentManagerCDCTest is flaky
> ---
>
> Key: CASSANDRA-14195
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14195
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Ariel Weisberg
>Assignee: Jay Zhuang
>Priority: Minor
>
> This fails fairly reliably in CircleCI and in a few minutes if you run it in
> a loop on a MacOS laptop.
> I see two failures.
> {noformat}
> [junit] Testcase:
> testRetainLinkOnDiscardCDC(org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest):
>Caused an ERROR
> [junit] Rejecting mutation to keyspace cql_test_keyspace. Free up space
> in build/test/cassandra/cdc_raw:0 by processing CDC logs.
> [junit] org.apache.cassandra.exceptions.CDCWriteException: Rejecting
> mutation to keyspace cql_test_keyspace. Free up space in
> build/test/cassandra/cdc_raw:0 by processing CDC logs.
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC.throwIfForbidden(CommitLogSegmentManagerCDC.java:136)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC.allocate(CommitLogSegmentManagerCDC.java:108)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:272)
> [junit] at
> org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:604)
> [junit] at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:481)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:191)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:196)
> [junit] at org.apache.cassandra.db.Mutation.apply(Mutation.java:205)
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testRetainLinkOnDiscardCDC(CommitLogSegmentManagerCDCTest.java:256)
> {noformat}
> and
> {noformat}
> [junit] Testcase:
> testCompletedFlag(org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest):
> FAILED
> [junit] Index file not written:
> build/test/cassandra/cdc_raw:0/CommitLog-7-1517005121474_cdc.idx
> [junit] junit.framework.AssertionFailedError: Index file not written:
> build/test/cassandra/cdc_raw:0/CommitLog-7-1517005121474_cdc.idx
> [junit] at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testCompletedFlag(CommitLogSegmentManagerCDCTest.java:210)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14141) Enable CDC unittest
[
https://issues.apache.org/jira/browse/CASSANDRA-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324257#comment-16324257
]
Joshua McKenzie commented on CASSANDRA-14141:
-
Just a quick note: I wouldn't remove or stop running the {{test-cdc}} target on
ci. The reason I kept these completely separate was to fully exercise the rest
of the testing infrastructure with cdc enabled to confirm no regression / data
loss in the CommitLog from the changed cdc code. Running the unit tests
specific to cdc on a regular {{ant test}} run is fine as a smoke test but
shouldn't serve as a replacement for running the entire test suite against the
changed logic.
> Enable CDC unittest
> ---
>
> Key: CASSANDRA-14141
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14141
> Project: Cassandra
> Issue Type: Test
> Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Labels: CDC, testing
> Fix For: 3.11.2, 4.0
>
>
> Follow up for CASSANDRA-14066
> 2 CDC unittests are skipped in the normal test run, it has to be {{$ ant
> test-cdc}} to run the cdc test.
> The fix enables them in normal {{$ ant test}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13006) Disable automatic heap dumps on OOM error
[ https://issues.apache.org/jira/browse/CASSANDRA-13006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-13006: Status: Ready to Commit (was: Patch Available) > Disable automatic heap dumps on OOM error > - > > Key: CASSANDRA-13006 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13006 > Project: Cassandra > Issue Type: Bug > Components: Configuration >Reporter: anmols >Assignee: Benjamin Lerer >Priority: Minor > Attachments: 13006-3.0.9.txt > > > With CASSANDRA-9861, a change was added to enable collecting heap dumps by > default if the process encountered an OOM error. These heap dumps are stored > in the Apache Cassandra home directory unless configured otherwise (see > [Cassandra Support > Document|https://support.datastax.com/hc/en-us/articles/204225959-Generating-and-Analyzing-Heap-Dumps] > for this feature). > > The creation and storage of heap dumps aides debugging and investigative > workflows, but is not be desirable for a production environment where these > heap dumps may occupy a large amount of disk space and require manual > intervention for cleanups. > > Managing heap dumps on out of memory errors and configuring the paths for > these heap dumps are available as JVM options in JVM. The current behavior > conflicts with the Boolean JVM flag HeapDumpOnOutOfMemoryError. > > A patch can be proposed here that would make the heap dump on OOM error honor > the HeapDumpOnOutOfMemoryError flag. Users who would want to still generate > heap dumps on OOM errors can set the -XX:+HeapDumpOnOutOfMemoryError JVM > option. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13006) Disable automatic heap dumps on OOM error
[
https://issues.apache.org/jira/browse/CASSANDRA-13006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260876#comment-16260876
]
Joshua McKenzie commented on CASSANDRA-13006:
-
Sorry for the delay - didn't realize I was reviewer on this.
Feedback follows:
general feedback:
- In our launch scripts, why have a separate JVM_ON_OUT_OF_MEMORY_ERROR_OPT
instead of nesting it inside JVM_OPTS? The latter seems more consistent w/other
configuration approaches. This would also clean up launching branch logic a
touch.
nits:
- Not sure why StartupChecks.checkOutOfMemoryHandling() is scoped to protected.
Should be able to be private.
- remove added whitespace at top of bin/cassandra.sh
- misspelling: {code} # uncomment the prefered option {code} should be
{{preferred}}
- Same holds for conf/cassandra-env.sh
- Having parseJavaVersion and parseUpdate both take no arguments and
System.getProperty("java.version") would be a cleaner interface. This isn't
critical path in any way so the duplicate calls to getProperty would be no real
issue.
- "jmcd" should read "jcmd" in comments in HeapUtils.java, line 55
- Since HeapUtils is the only call-site for {{logProcessOutput}}, that can be
inlined if you're in the mood for it.
Basically a bag of trivial things - looks good on the whole. +1 with the above
nits dealt with as you see fit.
> Disable automatic heap dumps on OOM error
> -
>
> Key: CASSANDRA-13006
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13006
> Project: Cassandra
> Issue Type: Bug
> Components: Configuration
>Reporter: anmols
>Assignee: Benjamin Lerer
>Priority: Minor
> Attachments: 13006-3.0.9.txt
>
>
> With CASSANDRA-9861, a change was added to enable collecting heap dumps by
> default if the process encountered an OOM error. These heap dumps are stored
> in the Apache Cassandra home directory unless configured otherwise (see
> [Cassandra Support
> Document|https://support.datastax.com/hc/en-us/articles/204225959-Generating-and-Analyzing-Heap-Dumps]
> for this feature).
>
> The creation and storage of heap dumps aides debugging and investigative
> workflows, but is not be desirable for a production environment where these
> heap dumps may occupy a large amount of disk space and require manual
> intervention for cleanups.
>
> Managing heap dumps on out of memory errors and configuring the paths for
> these heap dumps are available as JVM options in JVM. The current behavior
> conflicts with the Boolean JVM flag HeapDumpOnOutOfMemoryError.
>
> A patch can be proposed here that would make the heap dump on OOM error honor
> the HeapDumpOnOutOfMemoryError flag. Users who would want to still generate
> heap dumps on OOM errors can set the -XX:+HeapDumpOnOutOfMemoryError JVM
> option.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13530) GroupCommitLogService
[ https://issues.apache.org/jira/browse/CASSANDRA-13530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251653#comment-16251653 ] Joshua McKenzie commented on CASSANDRA-13530: - bq. This is the cowardly way of testing the different modes and their replayability rather than reworking the commit log as a whole (as Joshua McKenzie alludes to), but it seems like that's a larger issue to tackle (on a different ticket). There is no shame in Getting Things Done. :) But yeah, the complexity smell around the CommitLog as a whole is my personal white whale. > GroupCommitLogService > - > > Key: CASSANDRA-13530 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13530 > Project: Cassandra > Issue Type: Improvement >Reporter: Yuji Ito >Assignee: Yuji Ito > Fix For: 2.2.x, 3.0.x, 3.11.x > > Attachments: GuavaRequestThread.java, MicroRequestThread.java, > groupAndBatch.png, groupCommit22.patch, groupCommit30.patch, > groupCommit3x.patch, groupCommitLog_noSerial_result.xlsx, > groupCommitLog_result.xlsx > > > I propose a new CommitLogService, GroupCommitLogService, to improve the > throughput when lots of requests are received. > It improved the throughput by maximum 94%. > I'd like to discuss about this CommitLogService. > Currently, we can select either 2 CommitLog services; Periodic and Batch. > In Periodic, we might lose some commit log which hasn't written to the disk. > In Batch, we can write commit log to the disk every time. The size of commit > log to write is too small (< 4KB). When high concurrency, these writes are > gathered and persisted to the disk at once. But, when insufficient > concurrency, many small writes are issued and the performance decreases due > to the latency of the disk. Even if you use SSD, processes of many IO > commands decrease the performance. > GroupCommitLogService writes some commitlog to the disk at once. > The patch adds GroupCommitLogService (It is enabled by setting > `commitlog_sync` and `commitlog_sync_group_window_in_ms` in cassandra.yaml). > The difference from Batch is just only waiting for the semaphore. > By waiting for the semaphore, some writes for commit logs are executed at the > same time. > In GroupCommitLogService, the latency becomes worse if the there is no > concurrency. > I measured the performance with my microbench (MicroRequestThread.java) by > increasing the number of threads.The cluster has 3 nodes (Replication factor: > 3). Each nodes is AWS EC2 m4.large instance + 200IOPS io1 volume. > The result is as below. The GroupCommitLogService with 10ms window improved > update with Paxos by 94% and improved select with Paxos by 76%. > h6. SELECT / sec > ||\# of threads||Batch 2ms||Group 10ms|| > |1|192|103| > |2|163|212| > |4|264|416| > |8|454|800| > |16|744|1311| > |32|1151|1481| > |64|1767|1844| > |128|2949|3011| > |256|4723|5000| > h6. UPDATE / sec > ||\# of threads||Batch 2ms||Group 10ms|| > |1|45|26| > |2|39|51| > |4|58|102| > |8|102|198| > |16|167|213| > |32|289|295| > |64|544|548| > |128|1046|1058| > |256|2020|2061| -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
