[jira] [Updated] (CASSANDRA-14008) RTs at index boundaries in 2.x sstables can create unexpected CQL row in 3.x
[ https://issues.apache.org/jira/browse/CASSANDRA-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-14008: --- Reviewer: (was: Aleksey Yeschenko) > RTs at index boundaries in 2.x sstables can create unexpected CQL row in 3.x > > > Key: CASSANDRA-14008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14008 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Labels: correctness > Fix For: 3.0.x, 3.11.x > > > In 2.1/2.2, it is possible for a range tombstone that isn't a row deletion > and isn't a complex deletion to appear between two cells with the same > clustering. The 8099 legacy code incorrectly treats the two (non-RT) cells as > two distinct CQL rows, despite having the same clustering prefix. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14008) RTs at index boundaries in 2.x sstables can create unexpected CQL row in 3.x
[ https://issues.apache.org/jira/browse/CASSANDRA-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-14008: --- Reviewer: Aleksey Yeschenko > RTs at index boundaries in 2.x sstables can create unexpected CQL row in 3.x > > > Key: CASSANDRA-14008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14008 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Labels: correctness > Fix For: 3.0.x, 3.11.x > > > In 2.1/2.2, it is possible for a range tombstone that isn't a row deletion > and isn't a complex deletion to appear between two cells with the same > clustering. The 8099 legacy code incorrectly treats the two (non-RT) cells as > two distinct CQL rows, despite having the same clustering prefix. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13475) Pluggable storage engine design
[ https://issues.apache.org/jira/browse/CASSANDRA-13475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dikang Gu updated CASSANDRA-13475: -- Description: In this jira, we discuss how to make Cassandra's storage engine to be pluggable. We will discuss the scope, expectation, and guideline for this project, as well as a detailed design so that we can create sub tasks for each small project. Here is a design quip we are currently working on: https://quip.com/bhw5ABUCi3co was: In this jira, we discuss how to make Cassandra's storage engine to be pluggable. In order to support pluggable storage engine, we need to define a unified interface/API, which can allow us to plug in different storage engines for different requirements. Here is a design quip we are currently working on: https://quip.com/bhw5ABUCi3co In very high level, the storage engine interface should include APIs to: 1. Apply update into the engine. 2. Query data from the engine. 3. Stream data in/out to/from the engine. 4. Table operations, like create/drop/truncate a table, etc. 5. Various stats about the engine. I create this ticket to start the discussions about the interface. > Pluggable storage engine design > --- > > Key: CASSANDRA-13475 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13475 > Project: Cassandra > Issue Type: Sub-task >Reporter: Dikang Gu >Assignee: Dikang Gu > > In this jira, we discuss how to make Cassandra's storage engine to be > pluggable. We will discuss the scope, expectation, and guideline for this > project, as well as a detailed design so that we can create sub tasks for > each small project. > Here is a design quip we are currently working on: > https://quip.com/bhw5ABUCi3co -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13475) Pluggable storage engine design
[ https://issues.apache.org/jira/browse/CASSANDRA-13475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dikang Gu updated CASSANDRA-13475: -- Description: In this jira, we discuss how to make Cassandra's storage engine to be pluggable. In order to support pluggable storage engine, we need to define a unified interface/API, which can allow us to plug in different storage engines for different requirements. Here is a design quip we are currently working on: https://quip.com/bhw5ABUCi3co In very high level, the storage engine interface should include APIs to: 1. Apply update into the engine. 2. Query data from the engine. 3. Stream data in/out to/from the engine. 4. Table operations, like create/drop/truncate a table, etc. 5. Various stats about the engine. I create this ticket to start the discussions about the interface. was: In order to support pluggable storage engine, we need to define a unified interface/API, which can allow us to plug in different storage engines for different requirements. Here is a design quip we are currently working on: https://quip.com/bhw5ABUCi3co In very high level, the storage engine interface should include APIs to: 1. Apply update into the engine. 2. Query data from the engine. 3. Stream data in/out to/from the engine. 4. Table operations, like create/drop/truncate a table, etc. 5. Various stats about the engine. I create this ticket to start the discussions about the interface. > Pluggable storage engine design > --- > > Key: CASSANDRA-13475 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13475 > Project: Cassandra > Issue Type: Sub-task >Reporter: Dikang Gu >Assignee: Dikang Gu > > In this jira, we discuss how to make Cassandra's storage engine to be > pluggable. > In order to support pluggable storage engine, we need to define a unified > interface/API, which can allow us to plug in different storage engines for > different requirements. > Here is a design quip we are currently working on: > https://quip.com/bhw5ABUCi3co > In very high level, the storage engine interface should include APIs to: > 1. Apply update into the engine. > 2. Query data from the engine. > 3. Stream data in/out to/from the engine. > 4. Table operations, like create/drop/truncate a table, etc. > 5. Various stats about the engine. > I create this ticket to start the discussions about the interface. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13475) Pluggable storage engine design
[ https://issues.apache.org/jira/browse/CASSANDRA-13475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dikang Gu updated CASSANDRA-13475: -- Summary: Pluggable storage engine design (was: First version of pluggable storage engine API.) > Pluggable storage engine design > --- > > Key: CASSANDRA-13475 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13475 > Project: Cassandra > Issue Type: Sub-task >Reporter: Dikang Gu >Assignee: Dikang Gu > > In order to support pluggable storage engine, we need to define a unified > interface/API, which can allow us to plug in different storage engines for > different requirements. > Here is a design quip we are currently working on: > https://quip.com/bhw5ABUCi3co > In very high level, the storage engine interface should include APIs to: > 1. Apply update into the engine. > 2. Query data from the engine. > 3. Stream data in/out to/from the engine. > 4. Table operations, like create/drop/truncate a table, etc. > 5. Various stats about the engine. > I create this ticket to start the discussions about the interface. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13475) First version of pluggable storage engine API.
[ https://issues.apache.org/jira/browse/CASSANDRA-13475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247935#comment-16247935 ] Dikang Gu commented on CASSANDRA-13475: --- I agree it's a huge task, and there will be significant effort needed to refactor existing storage engine. IMO, this is the cost we need to pay, in order to make Cassandra a world class database. It helps us to estimate the resources and timeline of this project, but should be not the excuse that we can not do it. RocksDB is not within the scope of this particular "pluggable storage engine" project, it's the motivation why we want the pluggable storage engine so much. Cassandra's read performance, especially P99 read latency is not great, while RocksDB is a solid and well tuned LSM engine. We have put multiple engineers, spent 6+ months to prove that we can get huge performance gains, by leveraging RocksDB as the storage engine. And we have deployed it in our production environment, under real traffic. So we are committed to the pluggable storage engine project, to avoid a fork of Cassandra within Instagram/Facebook. Back to step 1, the scope/expectation/guideline of the project, I agree with Blake, Aleksey and Sylvain, we want to do it in right way, definitely not a hack in the database. I think we are on the same page of the high quality of the refactoring, and I'm very happy to discuss more details on step 1. I chatted with Nate, Blake and Jon offline, I will convert the design quip to a more formal design doc, and we can discuss there. Also, I will change the title of this jira, the "First version of pluggable storage engine API." is probably a bit mis-leading at this moment. > First version of pluggable storage engine API. > -- > > Key: CASSANDRA-13475 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13475 > Project: Cassandra > Issue Type: Sub-task >Reporter: Dikang Gu >Assignee: Dikang Gu > > In order to support pluggable storage engine, we need to define a unified > interface/API, which can allow us to plug in different storage engines for > different requirements. > Here is a design quip we are currently working on: > https://quip.com/bhw5ABUCi3co > In very high level, the storage engine interface should include APIs to: > 1. Apply update into the engine. > 2. Query data from the engine. > 3. Stream data in/out to/from the engine. > 4. Table operations, like create/drop/truncate a table, etc. > 5. Various stats about the engine. > I create this ticket to start the discussions about the interface. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13307) The specification of protocol version in cqlsh means the python driver doesn't automatically downgrade protocol version.
[ https://issues.apache.org/jira/browse/CASSANDRA-13307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247897#comment-16247897 ] Sven Ludwig commented on CASSANDRA-13307: - Is this fix already in a version available via pip from https://pypi.python.org/pypi/cqlsh/ ? If not, can you publish it there? > The specification of protocol version in cqlsh means the python driver > doesn't automatically downgrade protocol version. > > > Key: CASSANDRA-13307 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13307 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Matt Byrd >Assignee: Matt Byrd >Priority: Minor > Labels: doc-impacting > Fix For: 3.11.0, 4.0 > > > Hi, > Looks like we've regressed on the issue described in: > https://issues.apache.org/jira/browse/CASSANDRA-9467 > In that we're no longer able to connect from newer cqlsh versions > (e.g trunk) to older versions of Cassandra with a lower version of the > protocol (e.g 2.1 with protocol version 3) > The problem seems to be that we're relying on the ability for the client to > automatically downgrade protocol version implemented in Cassandra here: > https://issues.apache.org/jira/browse/CASSANDRA-12838 > and utilised in the python client here: > https://datastax-oss.atlassian.net/browse/PYTHON-240 > The problem however comes when we implemented: > https://datastax-oss.atlassian.net/browse/PYTHON-537 > "Don't downgrade protocol version if explicitly set" > (included when we bumped from 3.5.0 to 3.7.0 of the python driver as part of > fixing: https://issues.apache.org/jira/browse/CASSANDRA-11534) > Since we do explicitly specify the protocol version in the bin/cqlsh.py. > I've got a patch which just adds an option to explicitly specify the protocol > version (for those who want to do that) and then otherwise defaults to not > setting the protocol version, i.e using the protocol version from the client > which we ship, which should by default be the same protocol as the server. > Then it should downgrade gracefully as was intended. > Let me know if that seems reasonable. > Thanks, > Matt -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14008) RTs at index boundaries in 2.x sstables create unexpected CQL row in 3.x
[ https://issues.apache.org/jira/browse/CASSANDRA-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-14008: --- Labels: correctness (was: ) > RTs at index boundaries in 2.x sstables create unexpected CQL row in 3.x > > > Key: CASSANDRA-14008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14008 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Labels: correctness > Fix For: 3.0.x, 3.11.x > > > In 2.1/2.2, it is possible for a range tombstone that isn't a row deletion > and isn't a complex deletion to appear between two cells with the same > clustering. The 8099 legacy code incorrectly treats the two (non-RT) cells as > two distinct CQL rows, despite having the same clustering prefix. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14008) RTs at index boundaries in 2.x sstables can create unexpected CQL row in 3.x
[ https://issues.apache.org/jira/browse/CASSANDRA-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-14008: --- Summary: RTs at index boundaries in 2.x sstables can create unexpected CQL row in 3.x (was: RTs at index boundaries in 2.x sstables create unexpected CQL row in 3.x) > RTs at index boundaries in 2.x sstables can create unexpected CQL row in 3.x > > > Key: CASSANDRA-14008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14008 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Labels: correctness > Fix For: 3.0.x, 3.11.x > > > In 2.1/2.2, it is possible for a range tombstone that isn't a row deletion > and isn't a complex deletion to appear between two cells with the same > clustering. The 8099 legacy code incorrectly treats the two (non-RT) cells as > two distinct CQL rows, despite having the same clustering prefix. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14008) RTs at index boundaries in 2.x sstables create unexpected CQL row in 3.x
[ https://issues.apache.org/jira/browse/CASSANDRA-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-14008: --- Description: In 2.1/2.2, it is possible for a range tombstone that isn't a row deletion and isn't a complex deletion to appear between two cells with the same clustering. The 8099 legacy code incorrectly treats the two (non-RT) cells as two distinct CQL rows, despite having the same clustering prefix. (was: TBD) > RTs at index boundaries in 2.x sstables create unexpected CQL row in 3.x > > > Key: CASSANDRA-14008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14008 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Fix For: 3.0.x, 3.11.x > > > In 2.1/2.2, it is possible for a range tombstone that isn't a row deletion > and isn't a complex deletion to appear between two cells with the same > clustering. The 8099 legacy code incorrectly treats the two (non-RT) cells as > two distinct CQL rows, despite having the same clustering prefix. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14008) RTs at index boundaries in 2.x sstables create unexpected CQL row in 3.x
[ https://issues.apache.org/jira/browse/CASSANDRA-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-14008: --- Summary: RTs at index boundaries in 2.x sstables create unexpected CQL row in 3.x (was: TBD) > RTs at index boundaries in 2.x sstables create unexpected CQL row in 3.x > > > Key: CASSANDRA-14008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14008 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Fix For: 3.0.x, 3.11.x > > > TBD -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13561) Purge TTL on expiration
[ https://issues.apache.org/jira/browse/CASSANDRA-13561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Whang updated CASSANDRA-13561: - Resolution: Duplicate Status: Resolved (was: Patch Available) > Purge TTL on expiration > --- > > Key: CASSANDRA-13561 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13561 > Project: Cassandra > Issue Type: New Feature >Reporter: Andrew Whang >Assignee: Andrew Whang >Priority: Minor > Fix For: 4.0 > > > Tables with mostly TTL columns tend to suffer from high droppable tombstone > ratio, which results in higher read latency, cpu utilization, and disk usage. > Expired TTL data become tombstones, and the nature of purging tombstones > during compaction (due to checking for overlapping SSTables) make them > susceptible to surviving much longer than expected. A table option to purge > TTL on expiration would address this issue, by preventing them from becoming > tombstones. A boolean purge_ttl_on_expiration table setting would allow users > to easily turn the feature on or off. > Being more aggressive with gc_grace could also address the problem of long > lasting tombstones, but that would affect tombstones from deletes as well. > Even if a purged [expired] cell is revived via repair from a node that hasn't > yet compacted away the cell, it would be revived as an expiring cell with the > same localDeletionTime, so reads should properly handle them. As well, it > would be purged in the next compaction. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13958) [CQL] Inconsistent handling double dollar sign for strings
[
https://issues.apache.org/jira/browse/CASSANDRA-13958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247815#comment-16247815
]
Michał Szczygieł commented on CASSANDRA-13958:
--
by
bq. I think that is the only viable and probably non-surprising way.
you mean my suggestion to accept only one one either side, or something else?
> [CQL] Inconsistent handling double dollar sign for strings
> --
>
> Key: CASSANDRA-13958
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13958
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Hugo Picado
>Assignee: Michał Szczygieł
>Priority: Minor
>
> Double dollar signs is a [built-in method for escaping columns that may
> contain single quotes in its
> content](https://docs.datastax.com/en/cql/3.3/cql/cql_reference/escape_char_r.html).
> The way this is handled however is not consistent, in the sense that it
> allows for $ to appear in the middle of the string but not in the last char.
> *Examples:*
> Valid: insert into users(id, name) values(1, $$john$$)
> Inserts the string *john*
> Valid: insert into users(id, name) values(1, $$jo$hn$$)
> Inserts the string *jo$hn*
> Valid: insert into users(id, name) values(1, $$$john$$)
> Inserts the string *$john*
> Invalid: insert into users(id, name) values(1, $$john$$$)
> Fails with:
> {code}
> Invalid syntax at line 1, char 48
> insert into users(id, name) values(1, $$john$$$);
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13475) First version of pluggable storage engine API.
[ https://issues.apache.org/jira/browse/CASSANDRA-13475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247780#comment-16247780 ] Joshua McKenzie commented on CASSANDRA-13475: - bq. I'd say it's at the bare minimum a full-time one man-year project assuming a solid engineer that is pretty familiar with the code base to start with Given it took you, arguably one of the most knowledgeable engineers on the project since the start, a year and a half just to _refactor the Storage Engine_, refactoring even "just" the the inter-connected static state and tracking down and plugging sufficient abstraction leaks, not to mention invisible reliance on side effects / performance implications of the current formats, for things that touch that Storage Engine to make it safe to have it be pluggable... Yeah, I'd be super impressed if a single person working full-time got to a deliverable place in two years TBH. It takes a hell of a lot of work and deliberation to unwind a decade's worth of code-base debt to where making a change like this wouldn't be super high risk. bq. Don't get me wrong, I'd be very happy to see someone start to tackle that first step seriously, I think it's actually important for the project moving forward, but we're imo a long long way from being in a state where we can start to talk seriously about having pluggable storage engine in a clean way. I second that. This is something we've talked about extensively for years but nobody has ever really been able to start chewing on it in an incremental way. > First version of pluggable storage engine API. > -- > > Key: CASSANDRA-13475 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13475 > Project: Cassandra > Issue Type: Sub-task >Reporter: Dikang Gu >Assignee: Dikang Gu > > In order to support pluggable storage engine, we need to define a unified > interface/API, which can allow us to plug in different storage engines for > different requirements. > Here is a design quip we are currently working on: > https://quip.com/bhw5ABUCi3co > In very high level, the storage engine interface should include APIs to: > 1. Apply update into the engine. > 2. Query data from the engine. > 3. Stream data in/out to/from the engine. > 4. Table operations, like create/drop/truncate a table, etc. > 5. Various stats about the engine. > I create this ticket to start the discussions about the interface. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13475) First version of pluggable storage engine API.
[ https://issues.apache.org/jira/browse/CASSANDRA-13475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247651#comment-16247651 ] Sylvain Lebresne commented on CASSANDRA-13475: -- To chime in a bit here and for what it's worth, I concur with Blake's sentiment than a 'pluggable storage engine' is basically not imo realistic at this point. The Cassandra is everything but modular, we've never really be terribly careful in isolating the different parts of the code behind clearly defined and well isolated interfaces/APIs. Why that is can probably be debated for hours on end, and it's a big code-base so there is certainly parts that are much better than others. But on the whole, I'm not sure how one could qualify our story around modularity and abstraction of high level concepts in other terms than "it's a mess". So, and I'm kind of paraphrasing Blake here, I do feel that talking about a pluggable storage API today is completely missing step 1, which is a pretty massive refactor of the code to modularize and abstract much more cleanly the different part of the code base. Trying to bolt a pluggable storage API on our current mess without that first step would, in my professional opinion, but a terrible mistake for the project: I suspect it'll create a maintenance mess while having minuscule changes to bear real fruits. And I hope no-one is underestimating that first step: if I had to guess, and assuming we're talking about doing this is in a somewhat incremental way so it doesn't disturb all other dev in the meantime, I'd say it's _at the bare minimum_ a full-time one man-year project assuming a solid engineer that is pretty familiar with the code base to start with. I'd be of course the first to admit I suck at that kind of estimation so I'm probably way off, but I also can't remember the last time I _under_estimated something like that. Don't get me wrong, I'd be very happy to see someone start to tackle that first step seriously, I think it's actually important for the project moving forward, but we're imo a long long way from being in a state where we can start to talk seriously about having pluggable storage engine in a clean way. > First version of pluggable storage engine API. > -- > > Key: CASSANDRA-13475 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13475 > Project: Cassandra > Issue Type: Sub-task >Reporter: Dikang Gu >Assignee: Dikang Gu > > In order to support pluggable storage engine, we need to define a unified > interface/API, which can allow us to plug in different storage engines for > different requirements. > Here is a design quip we are currently working on: > https://quip.com/bhw5ABUCi3co > In very high level, the storage engine interface should include APIs to: > 1. Apply update into the engine. > 2. Query data from the engine. > 3. Stream data in/out to/from the engine. > 4. Table operations, like create/drop/truncate a table, etc. > 5. Various stats about the engine. > I create this ticket to start the discussions about the interface. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14008) TBD
Jeff Jirsa created CASSANDRA-14008: -- Summary: TBD Key: CASSANDRA-14008 URL: https://issues.apache.org/jira/browse/CASSANDRA-14008 Project: Cassandra Issue Type: Bug Components: Core Reporter: Jeff Jirsa Assignee: Jeff Jirsa Fix For: 3.0.x, 3.11.x TBD -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13958) [CQL] Inconsistent handling double dollar sign for strings
[
https://issues.apache.org/jira/browse/CASSANDRA-13958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247337#comment-16247337
]
Robert Stupp commented on CASSANDRA-13958:
--
I've played around with various forms of pg-style strings in CASSANDRA-7769
(the ticket that introduced this).
TL;DR there is [a
form|https://issues.apache.org/jira/browse/CASSANDRA-7769?focusedCommentId=14120240=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14120240]
that can handle all this, but making that work in cqlsh turned out to be super
difficult. TBH, I think that is the only viable and probably non-surprising way.
> [CQL] Inconsistent handling double dollar sign for strings
> --
>
> Key: CASSANDRA-13958
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13958
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Hugo Picado
>Assignee: Michał Szczygieł
>Priority: Minor
>
> Double dollar signs is a [built-in method for escaping columns that may
> contain single quotes in its
> content](https://docs.datastax.com/en/cql/3.3/cql/cql_reference/escape_char_r.html).
> The way this is handled however is not consistent, in the sense that it
> allows for $ to appear in the middle of the string but not in the last char.
> *Examples:*
> Valid: insert into users(id, name) values(1, $$john$$)
> Inserts the string *john*
> Valid: insert into users(id, name) values(1, $$jo$hn$$)
> Inserts the string *jo$hn*
> Valid: insert into users(id, name) values(1, $$$john$$)
> Inserts the string *$john*
> Invalid: insert into users(id, name) values(1, $$john$$$)
> Fails with:
> {code}
> Invalid syntax at line 1, char 48
> insert into users(id, name) values(1, $$john$$$);
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13948) Reload compaction strategies when JBOD disk boundary changes
[
https://issues.apache.org/jira/browse/CASSANDRA-13948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247193#comment-16247193
]
Marcus Eriksson commented on CASSANDRA-13948:
-
Just had a first pass of the commits, and in general it looks good, I added a
few comments on github
In general it feels a bit inconsistent about when it calls the
{{CSM#maybeReload()}} - with 13215 in, perhaps we could just always call it as
it will be cheap? I'll do a more thorough review of that last commit once it
has been rebased on top of 13215
> Reload compaction strategies when JBOD disk boundary changes
>
>
> Key: CASSANDRA-13948
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13948
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Paulo Motta
>Assignee: Paulo Motta
> Fix For: 3.11.x, 4.x
>
> Attachments: debug.log, threaddump-cleanup.txt, threaddump.txt,
> trace.log
>
>
> The thread dump below shows a race between an sstable replacement by the
> {{IndexSummaryRedistribution}} and
> {{AbstractCompactionTask.getNextBackgroundTask}}:
> {noformat}
> Thread 94580: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> -
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt()
> @bci=1, line=836 (Compiled frame)
> -
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.util.concurrent.locks.AbstractQueuedSynchronizer$Node,
> int) @bci=67, line=870 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(int)
> @bci=17, line=1199 (Compiled frame)
> - java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock() @bci=5,
> line=943 (Compiled frame)
> -
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleListChangedNotification(java.lang.Iterable,
> java.lang.Iterable) @bci=359, line=483 (Interpreted frame)
> -
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(org.apache.cassandra.notifications.INotification,
> java.lang.Object) @bci=53, line=555 (Interpreted frame)
> -
> org.apache.cassandra.db.lifecycle.Tracker.notifySSTablesChanged(java.util.Collection,
> java.util.Collection, org.apache.cassandra.db.compaction.OperationType,
> java.lang.Throwable) @bci=50, line=409 (Interpreted frame)
> -
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.doCommit(java.lang.Throwable)
> @bci=157, line=227 (Interpreted frame)
> -
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.commit(java.lang.Throwable)
> @bci=61, line=116 (Compiled frame)
> -
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.commit()
> @bci=2, line=200 (Interpreted frame)
> -
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish()
> @bci=5, line=185 (Interpreted frame)
> -
> org.apache.cassandra.io.sstable.IndexSummaryRedistribution.redistributeSummaries()
> @bci=559, line=130 (Interpreted frame)
> -
> org.apache.cassandra.db.compaction.CompactionManager.runIndexSummaryRedistribution(org.apache.cassandra.io.sstable.IndexSummaryRedistribution)
> @bci=9, line=1420 (Interpreted frame)
> -
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(org.apache.cassandra.io.sstable.IndexSummaryRedistribution)
> @bci=4, line=250 (Interpreted frame)
> -
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries()
> @bci=30, line=228 (Interpreted frame)
> - org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow()
> @bci=4, line=125 (Interpreted frame)
> - org.apache.cassandra.utils.WrappedRunnable.run() @bci=1, line=28
> (Interpreted frame)
> -
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run()
> @bci=4, line=118 (Compiled frame)
> - java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=511
> (Compiled frame)
> - java.util.concurrent.FutureTask.runAndReset() @bci=47, line=308 (Compiled
> frame)
> -
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> @bci=1, line=180 (Compiled frame)
> - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()
> @bci=37, line=294 (Compiled frame)
> -
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
> @bci=95, line=1149 (Compiled frame)
> - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=624
> (Interpreted frame)
> -
>
[jira] [Commented] (CASSANDRA-13006) Disable automatic heap dumps on OOM error
[
https://issues.apache.org/jira/browse/CASSANDRA-13006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247177#comment-16247177
]
Benjamin Lerer commented on CASSANDRA-13006:
I updated the patches for {{3.0}}, {{3.11}} and {{trunk}}. I replaced the use
of {{jmap}} by {{jcmd}}.
CI results look good for the unit tests.
> Disable automatic heap dumps on OOM error
> -
>
> Key: CASSANDRA-13006
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13006
> Project: Cassandra
> Issue Type: Bug
> Components: Configuration
>Reporter: anmols
>Assignee: Benjamin Lerer
>Priority: Minor
> Attachments: 13006-3.0.9.txt
>
>
> With CASSANDRA-9861, a change was added to enable collecting heap dumps by
> default if the process encountered an OOM error. These heap dumps are stored
> in the Apache Cassandra home directory unless configured otherwise (see
> [Cassandra Support
> Document|https://support.datastax.com/hc/en-us/articles/204225959-Generating-and-Analyzing-Heap-Dumps]
> for this feature).
>
> The creation and storage of heap dumps aides debugging and investigative
> workflows, but is not be desirable for a production environment where these
> heap dumps may occupy a large amount of disk space and require manual
> intervention for cleanups.
>
> Managing heap dumps on out of memory errors and configuring the paths for
> these heap dumps are available as JVM options in JVM. The current behavior
> conflicts with the Boolean JVM flag HeapDumpOnOutOfMemoryError.
>
> A patch can be proposed here that would make the heap dump on OOM error honor
> the HeapDumpOnOutOfMemoryError flag. Users who would want to still generate
> heap dumps on OOM errors can set the -XX:+HeapDumpOnOutOfMemoryError JVM
> option.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13006) Disable automatic heap dumps on OOM error
[ https://issues.apache.org/jira/browse/CASSANDRA-13006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-13006: --- Reviewer: Joshua McKenzie Fix Version/s: (was: 3.0.16) > Disable automatic heap dumps on OOM error > - > > Key: CASSANDRA-13006 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13006 > Project: Cassandra > Issue Type: Bug > Components: Configuration >Reporter: anmols >Assignee: Benjamin Lerer >Priority: Minor > Attachments: 13006-3.0.9.txt > > > With CASSANDRA-9861, a change was added to enable collecting heap dumps by > default if the process encountered an OOM error. These heap dumps are stored > in the Apache Cassandra home directory unless configured otherwise (see > [Cassandra Support > Document|https://support.datastax.com/hc/en-us/articles/204225959-Generating-and-Analyzing-Heap-Dumps] > for this feature). > > The creation and storage of heap dumps aides debugging and investigative > workflows, but is not be desirable for a production environment where these > heap dumps may occupy a large amount of disk space and require manual > intervention for cleanups. > > Managing heap dumps on out of memory errors and configuring the paths for > these heap dumps are available as JVM options in JVM. The current behavior > conflicts with the Boolean JVM flag HeapDumpOnOutOfMemoryError. > > A patch can be proposed here that would make the heap dump on OOM error honor > the HeapDumpOnOutOfMemoryError flag. Users who would want to still generate > heap dumps on OOM errors can set the -XX:+HeapDumpOnOutOfMemoryError JVM > option. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-8877) Ability to read the TTL and WRITE TIME of an element in a collection
[ https://issues.apache.org/jira/browse/CASSANDRA-8877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer reassigned CASSANDRA-8877: - Assignee: (was: Benjamin Lerer) > Ability to read the TTL and WRITE TIME of an element in a collection > > > Key: CASSANDRA-8877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8877 > Project: Cassandra > Issue Type: Improvement >Reporter: Drew Kutcharian >Priority: Minor > Fix For: 4.x > > > Currently it's possible to set the TTL and WRITE TIME of an element in a > collection using CQL, but there is no way to read them back. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
