[jira] [Updated] (CASSANDRA-14570) Improper default value of cdc_total_space_in_mb
[ https://issues.apache.org/jira/browse/CASSANDRA-14570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Zhuang updated CASSANDRA-14570: --- Labels: CDC (was: ) > Improper default value of cdc_total_space_in_mb > --- > > Key: CASSANDRA-14570 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14570 > Project: Cassandra > Issue Type: Bug > Components: Configuration >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > Labels: CDC > > The code for calculating cdc_total_space_in_mb in DatabaseDescriptor.java > does not reflect the latest architecture introduced by CASSANDRA-12148. > In short, cdc_total_space_in_mb should be equal or greater than > commitlog_total_space_in_mb; otherwise, the writes will fail when on-disk > commit logs size reaches the value of cdc_total_space_in_mb. For example, If > cdc_total_space_in_mb is 4096 and commitlog_total_space_in_mb is 8192, when > we enabled the cdc_enabled flag (even if we didn't enable cdc=true on any > table), when total size of commit logs reaches 4096 MB, there is the same of > amount of hard links in cdc_raw directory, that is, 4096 MB of cdc logs. Then > the CommitLogSegmentManagerCDC will be unable to process new CL segments. > (See CommitLogSegmentManagerCDC.processNewSegment method) > > I will attach patches later. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14570) Improper default value of cdc_total_space_in_mb
[ https://issues.apache.org/jira/browse/CASSANDRA-14570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546004#comment-16546004 ] Jay Zhuang commented on CASSANDRA-14570: Hi [~shichao.an], the size calculation in 3.11 branch is right, as CASSANDRA-12148 is only for 4.0. For the trunk change, it looks good to me. cc [~JoshuaMcKenzie]. > Improper default value of cdc_total_space_in_mb > --- > > Key: CASSANDRA-14570 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14570 > Project: Cassandra > Issue Type: Bug > Components: Configuration >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > The code for calculating cdc_total_space_in_mb in DatabaseDescriptor.java > does not reflect the latest architecture introduced by CASSANDRA-12148. > In short, cdc_total_space_in_mb should be equal or greater than > commitlog_total_space_in_mb; otherwise, the writes will fail when on-disk > commit logs size reaches the value of cdc_total_space_in_mb. For example, If > cdc_total_space_in_mb is 4096 and commitlog_total_space_in_mb is 8192, when > we enabled the cdc_enabled flag (even if we didn't enable cdc=true on any > table), when total size of commit logs reaches 4096 MB, there is the same of > amount of hard links in cdc_raw directory, that is, 4096 MB of cdc logs. Then > the CommitLogSegmentManagerCDC will be unable to process new CL segments. > (See CommitLogSegmentManagerCDC.processNewSegment method) > > I will attach patches later. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14570) Improper default value of cdc_total_space_in_mb
[ https://issues.apache.org/jira/browse/CASSANDRA-14570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Zhuang updated CASSANDRA-14570: --- Component/s: (was: Local Write-Read Paths) Configuration > Improper default value of cdc_total_space_in_mb > --- > > Key: CASSANDRA-14570 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14570 > Project: Cassandra > Issue Type: Bug > Components: Configuration >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > The code for calculating cdc_total_space_in_mb in DatabaseDescriptor.java > does not reflect the latest architecture introduced by CASSANDRA-12148. > In short, cdc_total_space_in_mb should be equal or greater than > commitlog_total_space_in_mb; otherwise, the writes will fail when on-disk > commit logs size reaches the value of cdc_total_space_in_mb. For example, If > cdc_total_space_in_mb is 4096 and commitlog_total_space_in_mb is 8192, when > we enabled the cdc_enabled flag (even if we didn't enable cdc=true on any > table), when total size of commit logs reaches 4096 MB, there is the same of > amount of hard links in cdc_raw directory, that is, 4096 MB of cdc logs. Then > the CommitLogSegmentManagerCDC will be unable to process new CL segments. > (See CommitLogSegmentManagerCDC.processNewSegment method) > > I will attach patches later. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14570) Improper default value of cdc_total_space_in_mb
[ https://issues.apache.org/jira/browse/CASSANDRA-14570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Zhuang updated CASSANDRA-14570: --- Reproduced In: 4.0 Status: Patch Available (was: Open) > Improper default value of cdc_total_space_in_mb > --- > > Key: CASSANDRA-14570 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14570 > Project: Cassandra > Issue Type: Bug > Components: Configuration >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > The code for calculating cdc_total_space_in_mb in DatabaseDescriptor.java > does not reflect the latest architecture introduced by CASSANDRA-12148. > In short, cdc_total_space_in_mb should be equal or greater than > commitlog_total_space_in_mb; otherwise, the writes will fail when on-disk > commit logs size reaches the value of cdc_total_space_in_mb. For example, If > cdc_total_space_in_mb is 4096 and commitlog_total_space_in_mb is 8192, when > we enabled the cdc_enabled flag (even if we didn't enable cdc=true on any > table), when total size of commit logs reaches 4096 MB, there is the same of > amount of hard links in cdc_raw directory, that is, 4096 MB of cdc logs. Then > the CommitLogSegmentManagerCDC will be unable to process new CL segments. > (See CommitLogSegmentManagerCDC.processNewSegment method) > > I will attach patches later. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14570) Improper default value of cdc_total_space_in_mb
[ https://issues.apache.org/jira/browse/CASSANDRA-14570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Zhuang reassigned CASSANDRA-14570: -- Assignee: Shichao An > Improper default value of cdc_total_space_in_mb > --- > > Key: CASSANDRA-14570 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14570 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > The code for calculating cdc_total_space_in_mb in DatabaseDescriptor.java > does not reflect the latest architecture introduced by CASSANDRA-12148. > In short, cdc_total_space_in_mb should be equal or greater than > commitlog_total_space_in_mb; otherwise, the writes will fail when on-disk > commit logs size reaches the value of cdc_total_space_in_mb. For example, If > cdc_total_space_in_mb is 4096 and commitlog_total_space_in_mb is 8192, when > we enabled the cdc_enabled flag (even if we didn't enable cdc=true on any > table), when total size of commit logs reaches 4096 MB, there is the same of > amount of hard links in cdc_raw directory, that is, 4096 MB of cdc logs. Then > the CommitLogSegmentManagerCDC will be unable to process new CL segments. > (See CommitLogSegmentManagerCDC.processNewSegment method) > > I will attach patches later. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14570) Improper default value of cdc_total_space_in_mb
[ https://issues.apache.org/jira/browse/CASSANDRA-14570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545909#comment-16545909 ] Shichao An commented on CASSANDRA-14570: Here's the patch for trunk and 3.11: ||Branch||uTest|| |[14570-trunk|https://github.com/shichao-an/cassandra/commits/14570-trunk]|[!https://circleci.com/gh/shichao-an/cassandra/tree/14570-trunk.svg?style=svg!|https://circleci.com/gh/shichao-an/cassandra/tree/14570-trunk]| |[14570-3.11|https://github.com/shichao-an/cassandra/commits/14570-3.11]|[!https://circleci.com/gh/shichao-an/cassandra/tree/14570-3.11.svg?style=svg!|https://circleci.com/gh/shichao-an/cassandra/tree/14570-3.11]| > Improper default value of cdc_total_space_in_mb > --- > > Key: CASSANDRA-14570 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14570 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Shichao An >Priority: Minor > > The code for calculating cdc_total_space_in_mb in DatabaseDescriptor.java > does not reflect the latest architecture introduced by CASSANDRA-12148. > In short, cdc_total_space_in_mb should be equal or greater than > commitlog_total_space_in_mb; otherwise, the writes will fail when on-disk > commit logs size reaches the value of cdc_total_space_in_mb. For example, If > cdc_total_space_in_mb is 4096 and commitlog_total_space_in_mb is 8192, when > we enabled the cdc_enabled flag (even if we didn't enable cdc=true on any > table), when total size of commit logs reaches 4096 MB, there is the same of > amount of hard links in cdc_raw directory, that is, 4096 MB of cdc logs. Then > the CommitLogSegmentManagerCDC will be unable to process new CL segments. > (See CommitLogSegmentManagerCDC.processNewSegment method) > > I will attach patches later. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14570) Improper default value of cdc_total_space_in_mb
Shichao An created CASSANDRA-14570: -- Summary: Improper default value of cdc_total_space_in_mb Key: CASSANDRA-14570 URL: https://issues.apache.org/jira/browse/CASSANDRA-14570 Project: Cassandra Issue Type: Bug Components: Local Write-Read Paths Reporter: Shichao An The code for calculating cdc_total_space_in_mb in DatabaseDescriptor.java does not reflect the latest architecture introduced by CASSANDRA-12148. In short, cdc_total_space_in_mb should be equal or greater than commitlog_total_space_in_mb; otherwise, the writes will fail when on-disk commit logs size reaches the value of cdc_total_space_in_mb. For example, If cdc_total_space_in_mb is 4096 and commitlog_total_space_in_mb is 8192, when we enabled the cdc_enabled flag (even if we didn't enable cdc=true on any table), when total size of commit logs reaches 4096 MB, there is the same of amount of hard links in cdc_raw directory, that is, 4096 MB of cdc logs. Then the CommitLogSegmentManagerCDC will be unable to process new CL segments. (See CommitLogSegmentManagerCDC.processNewSegment method) I will attach patches later. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14563) Add animalsniffer to build to ensure runtime jdk compatbility
[ https://issues.apache.org/jira/browse/CASSANDRA-14563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mck reassigned CASSANDRA-14563: --- Assignee: Sumanth Pasupuleti > Add animalsniffer to build to ensure runtime jdk compatbility > - > > Key: CASSANDRA-14563 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14563 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: mck >Assignee: Sumanth Pasupuleti >Priority: Minor > Labels: lhf > > Cassandra-2.2 still supports running on JDK1.7 > No tests check this though, as all build and test with JDK1.8 > Adding the ant animalsniffer task can check that jdk1.8 classes or methods > are not used accidentally. > ref: http://www.mojohaus.org/animal-sniffer/animal-sniffer/index.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14569) How to get the size of set in cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545813#comment-16545813 ] Rama Krishna commented on CASSANDRA-14569: -- Exception : Traceback (most recent call last): File "/charter/DEV/cassandra-3.0.15/bin/cqlsh.py", line 1301, in perform_simple_statement result = future.result() File "/charter/DEV/cassandra-3.0.15/bin/../lib/cassandra-driver-internal-only-3.7.1.post0-19c1603.zip/cassandra-driver-3.7.1.post0-19c1603/cassandra/cluster.py", line 3783, in result raise self._final_exception FunctionFailure: Error from server: code=1400 [User Defined Function failure] message="execution of 'test.getsize3[set]' failed: java.lang.NullPointerException > How to get the size of set in cassandra > --- > > Key: CASSANDRA-14569 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14569 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Rama Krishna >Priority: Blocker > Fix For: 3.0.x > > > Team, > > i have a table , which has a column defined as set , now i want to > calculate the size of that column , if want see if i have any null values for > that column . > please let me know if we have any way to find out all the null values for > that column . > Many thanks in advance . > > > Regards , > RK. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14569) How to get the size of set in cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545812#comment-16545812 ] Rama Krishna commented on CASSANDRA-14569: -- Team, do we have any updated , i tried creating a UDF , but it throws a java.lang.NullPointerException", as i have null values in mu table, how to overcome the Null pointer exception . below is the UDF created , CREATE FUNCTION test.getsize3(state set) CALLED ON NULL INPUT RETURNS int LANGUAGE java AS $$ return state.size(); $$; please let me know for any better inputs. > How to get the size of set in cassandra > --- > > Key: CASSANDRA-14569 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14569 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Rama Krishna >Priority: Blocker > Fix For: 3.0.x > > > Team, > > i have a table , which has a column defined as set , now i want to > calculate the size of that column , if want see if i have any null values for > that column . > please let me know if we have any way to find out all the null values for > that column . > Many thanks in advance . > > > Regards , > RK. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14568) Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages
[ https://issues.apache.org/jira/browse/CASSANDRA-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14568: -- Fix Version/s: (was: 3.11.x) (was: 3.0.x) 3.11.3 3.0.17 > Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages > > > Key: CASSANDRA-14568 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14568 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 3.0.17, 3.11.3 > > > In 2.1 and 2.2, row and complex deletions were represented as range > tombstones. LegacyLayout is our compatibility layer, that translates the > relevant RT patterns in 2.1/2.2 to row/complex deletions in 3.0, and vice > versa. Unfortunately, it does not handle the special case of static row > deletions, they are treated as regular row deletions. Since static rows are > themselves never directly deleted, the only issue is with collection > deletions. > Collection deletions in 2.1/2.2 were encoded as a range tombstone, consisting > of a sequence of the clustering keys’ data for the affected row, followed by > the bytes representing the name of the collection column. STATIC_CLUSTERING > contains zero clusterings, so by treating the deletion as for a regular row, > zero clusterings are written to precede the column name of the erased > collection, so the column name is written at position zero. > This can exhibit itself in at least two ways: > # If the type of your first clustering key is a variable width type, new > deletes will begin appearing covering the clustering key represented by the > column name. > ** If you have multiple clustering keys, you will receive a RT covering all > those rows with a matching first clustering key. > ** This RT will be valid as far as the system is concerned, and go > undetected unless there are outside data quality checks in place. > # Otherwise, an invalid size of data will be written to the clustering and > sent over the network to the 2.1 node. > ** The 2.1/2.2 node will handle this just fine, even though the record is > junk. Since it is a deletion covering impossible data, there will be no > user-API visible effect. But if received as a write from a 3.0 node, it will > dutifully persist the junk record. > ** The 3.0 node that originally sent this junk, may later coordinate a read > of the partition, and will notice a digest mismatch, read-repair and > serialize the junk to disk > ** The sstable containing this record is now corrupt; the deserialization > expects fixed-width data, but it encounters too many (or too few) bytes, and > is now at an incorrect position to read its structural information > ** (Alternatively when the 2.1 node is upgraded this will occur on eventual > compaction) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14567) CQL query returns different results in 2.2.5 and 3.0.15
[ https://issues.apache.org/jira/browse/CASSANDRA-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545652#comment-16545652 ] Dikang Gu edited comment on CASSANDRA-14567 at 7/16/18 7:31 PM: My main problem is about the semantics as well. According to the table schema definition (CLUSTERING ORDER BY (c1 DESC, c2 ASC)), the rows are sorted by following order on disk: 1, 10, 0, 1 1, 10, 10, 2 1, 1, 0, 3 1, 1, 10, 4 For query like "(c1, c2) >= (1, 10) AND (c1, c2) <= (10, 0)", if we interpret it to be "fetch all rows between row [1, 10, 0, * ] and [1, 1, 10, * ]", then it should return *all* the rows, which is the behavior in and before 2.2.5. But if we interpret the query to be "fetch all rows with (c1 >=1 and c2 >= 10) AND (c1 <= 10 and c2 <= 0)", (I have some problems to really understand this), which seems to be the behavior after 2.2.5, then it should return these two rows: *[Row[1, 10, 0, 1], Row[1, 1, 10, 4]].* My question is what should be the correct behavior here, and can we somehow differentiate these two behaviors, by having two different query grammars? was (Author: dikanggu): My main problem is about the semantics as well. According to the table schema definition (CLUSTERING ORDER BY (c1 DESC, c2 ASC)), the rows are sorted by following order on disk: 1, 10, 0, 1 1, 10, 10, 2 1, 1, 0, 3 1, 1, 10, 4 For query like "(c1, c2) >= (1, 10) AND (c1, c2) <= (10, 0)", if we interpret it to be "fetch all rows between row [1, 10, 0, * ] and [1, 1, 10, * ]", then it should return *all* the rows, which is the behavior in and before 2.2.5. But if we interpret the query to be "fetch all rows with (c1 >=1 and c2 >= 10) AND (c1 <= 10 and c2 <= 0)", (I have some problems to really understand this), which seems to be the behavior after 2.2.5, then it should return one two rows: *[Row[1, 10, 0, 1], Row[1, 1, 10, 4]].* My question is what should be the correct behavior here, and can we somehow differentiate these two behaviors, by having two different query grammars? > CQL query returns different results in 2.2.5 and 3.0.15 > --- > > Key: CASSANDRA-14567 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14567 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Dikang Gu >Priority: Major > > During our 2.2.5 to 3.0.15 upgrade, we find the a cql query returns different > results in 2.2.5 and 3.0.15, here is a unit test to reproduce it, > [https://gist.github.com/DikangGu/e538ed2de22b74e49b8dd43f7093a996] > > In C* 2.2.5, it returns all 4 rows: *[Row[1, 10, 0, 1], Row[1, 10, 10, 2], > Row[1, 1, 0, 3], Row[1, 1, 10, 4]]* > While in C* 3.0.15, it only returns 2 rows to client: *[Row[1, 10, 0, 1], > Row[1, 1, 10, 4]]* > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14567) CQL query returns different results in 2.2.5 and 3.0.15
[ https://issues.apache.org/jira/browse/CASSANDRA-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545652#comment-16545652 ] Dikang Gu commented on CASSANDRA-14567: --- My main problem is about the semantics as well. According to the table schema definition (CLUSTERING ORDER BY (c1 DESC, c2 ASC)), the rows are sorted by following order on disk: 1, 10, 0, 1 1, 10, 10, 2 1, 1, 0, 3 1, 1, 10, 4 For query like "(c1, c2) >= (1, 10) AND (c1, c2) <= (10, 0)", if we interpret it to be "fetch all rows between row [1, 10, 0, * ] and [1, 1, 10, * ]", then it should return *all* the rows, which is the behavior in and before 2.2.5. But if we interpret the query to be "fetch all rows with (c1 >=1 and c2 >= 10) AND (c1 <= 10 and c2 <= 0)", (I have some problems to really understand this), which seems to be the behavior after 2.2.5, then it should return one two rows: *[Row[1, 10, 0, 1], Row[1, 1, 10, 4]].* My question is what should be the correct behavior here, and can we somehow differentiate these two behaviors, by having two different query grammars? > CQL query returns different results in 2.2.5 and 3.0.15 > --- > > Key: CASSANDRA-14567 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14567 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Dikang Gu >Priority: Major > > During our 2.2.5 to 3.0.15 upgrade, we find the a cql query returns different > results in 2.2.5 and 3.0.15, here is a unit test to reproduce it, > [https://gist.github.com/DikangGu/e538ed2de22b74e49b8dd43f7093a996] > > In C* 2.2.5, it returns all 4 rows: *[Row[1, 10, 0, 1], Row[1, 10, 10, 2], > Row[1, 1, 0, 3], Row[1, 1, 10, 4]]* > While in C* 3.0.15, it only returns 2 rows to client: *[Row[1, 10, 0, 1], > Row[1, 1, 10, 4]]* > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14563) Add animalsniffer to build to ensure runtime jdk compatbility
[ https://issues.apache.org/jira/browse/CASSANDRA-14563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545594#comment-16545594 ] Sumanth Pasupuleti commented on CASSANDRA-14563: [~michaelsembwever] I would like to work on this ticket. Can you please assign > Add animalsniffer to build to ensure runtime jdk compatbility > - > > Key: CASSANDRA-14563 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14563 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: mck >Priority: Minor > Labels: lhf > > Cassandra-2.2 still supports running on JDK1.7 > No tests check this though, as all build and test with JDK1.8 > Adding the ant animalsniffer task can check that jdk1.8 classes or methods > are not used accidentally. > ref: http://www.mojohaus.org/animal-sniffer/animal-sniffer/index.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14556) Optimize streaming path in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545581#comment-16545581 ] Aleksey Yeschenko commented on CASSANDRA-14556: --- [~aweisberg] sure! > Optimize streaming path in Cassandra > > > Key: CASSANDRA-14556 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14556 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Major > Labels: Performance > Fix For: 4.x > > > During streaming, Cassandra reifies the sstables into objects. This creates > unnecessary garbage and slows down the whole streaming process as some > sstables can be transferred as a whole file rather than individual > partitions. The objective of the ticket is to detect when a whole sstable can > be transferred and skip the object reification. We can also use a zero-copy > path to avoid bringing data into user-space on both sending and receiving > side. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14556) Optimize streaming path in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14556: -- Reviewer: (was: Ariel Weisberg) > Optimize streaming path in Cassandra > > > Key: CASSANDRA-14556 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14556 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Major > Labels: Performance > Fix For: 4.x > > > During streaming, Cassandra reifies the sstables into objects. This creates > unnecessary garbage and slows down the whole streaming process as some > sstables can be transferred as a whole file rather than individual > partitions. The objective of the ticket is to detect when a whole sstable can > be transferred and skip the object reification. We can also use a zero-copy > path to avoid bringing data into user-space on both sending and receiving > side. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14556) Optimize streaming path in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14556: --- Reviewers: Aleksey Yeschenko, Ariel Weisberg (was: Aleksey Yeschenko) > Optimize streaming path in Cassandra > > > Key: CASSANDRA-14556 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14556 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Major > Labels: Performance > Fix For: 4.x > > > During streaming, Cassandra reifies the sstables into objects. This creates > unnecessary garbage and slows down the whole streaming process as some > sstables can be transferred as a whole file rather than individual > partitions. The objective of the ticket is to detect when a whole sstable can > be transferred and skip the object reification. We can also use a zero-copy > path to avoid bringing data into user-space on both sending and receiving > side. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14556) Optimize streaming path in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545576#comment-16545576 ] Ariel Weisberg commented on CASSANDRA-14556: [~iamaleksey] can you give it a final once over? > Optimize streaming path in Cassandra > > > Key: CASSANDRA-14556 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14556 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Major > Labels: Performance > Fix For: 4.x > > > During streaming, Cassandra reifies the sstables into objects. This creates > unnecessary garbage and slows down the whole streaming process as some > sstables can be transferred as a whole file rather than individual > partitions. The objective of the ticket is to detect when a whole sstable can > be transferred and skip the object reification. We can also use a zero-copy > path to avoid bringing data into user-space on both sending and receiving > side. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14556) Optimize streaming path in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14556: --- Reviewers: Aleksey Yeschenko > Optimize streaming path in Cassandra > > > Key: CASSANDRA-14556 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14556 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Major > Labels: Performance > Fix For: 4.x > > > During streaming, Cassandra reifies the sstables into objects. This creates > unnecessary garbage and slows down the whole streaming process as some > sstables can be transferred as a whole file rather than individual > partitions. The objective of the ticket is to detect when a whole sstable can > be transferred and skip the object reification. We can also use a zero-copy > path to avoid bringing data into user-space on both sending and receiving > side. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14567) CQL query returns different results in 2.2.5 and 3.0.15
[ https://issues.apache.org/jira/browse/CASSANDRA-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545556#comment-16545556 ] Jay Zhuang commented on CASSANDRA-14567: I think this is duplicated of CASSANDRA-7281. > CQL query returns different results in 2.2.5 and 3.0.15 > --- > > Key: CASSANDRA-14567 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14567 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Dikang Gu >Priority: Major > > During our 2.2.5 to 3.0.15 upgrade, we find the a cql query returns different > results in 2.2.5 and 3.0.15, here is a unit test to reproduce it, > [https://gist.github.com/DikangGu/e538ed2de22b74e49b8dd43f7093a996] > > In C* 2.2.5, it returns all 4 rows: *[Row[1, 10, 0, 1], Row[1, 10, 10, 2], > Row[1, 1, 0, 3], Row[1, 1, 10, 4]]* > While in C* 3.0.15, it only returns 2 rows to client: *[Row[1, 10, 0, 1], > Row[1, 1, 10, 4]]* > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14568) Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages
[ https://issues.apache.org/jira/browse/CASSANDRA-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-14568: - Resolution: Fixed Status: Resolved (was: Ready to Commit) > Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages > > > Key: CASSANDRA-14568 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14568 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 3.0.x, 3.11.x > > > In 2.1 and 2.2, row and complex deletions were represented as range > tombstones. LegacyLayout is our compatibility layer, that translates the > relevant RT patterns in 2.1/2.2 to row/complex deletions in 3.0, and vice > versa. Unfortunately, it does not handle the special case of static row > deletions, they are treated as regular row deletions. Since static rows are > themselves never directly deleted, the only issue is with collection > deletions. > Collection deletions in 2.1/2.2 were encoded as a range tombstone, consisting > of a sequence of the clustering keys’ data for the affected row, followed by > the bytes representing the name of the collection column. STATIC_CLUSTERING > contains zero clusterings, so by treating the deletion as for a regular row, > zero clusterings are written to precede the column name of the erased > collection, so the column name is written at position zero. > This can exhibit itself in at least two ways: > # If the type of your first clustering key is a variable width type, new > deletes will begin appearing covering the clustering key represented by the > column name. > ** If you have multiple clustering keys, you will receive a RT covering all > those rows with a matching first clustering key. > ** This RT will be valid as far as the system is concerned, and go > undetected unless there are outside data quality checks in place. > # Otherwise, an invalid size of data will be written to the clustering and > sent over the network to the 2.1 node. > ** The 2.1/2.2 node will handle this just fine, even though the record is > junk. Since it is a deletion covering impossible data, there will be no > user-API visible effect. But if received as a write from a 3.0 node, it will > dutifully persist the junk record. > ** The 3.0 node that originally sent this junk, may later coordinate a read > of the partition, and will notice a digest mismatch, read-repair and > serialize the junk to disk > ** The sstable containing this record is now corrupt; the deserialization > expects fixed-width data, but it encounters too many (or too few) bytes, and > is now at an incorrect position to read its structural information > ** (Alternatively when the 2.1 node is upgraded this will occur on eventual > compaction) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14569) How to get the size of set in cassandra
Rama Krishna created CASSANDRA-14569: Summary: How to get the size of set in cassandra Key: CASSANDRA-14569 URL: https://issues.apache.org/jira/browse/CASSANDRA-14569 Project: Cassandra Issue Type: Task Components: Build Reporter: Rama Krishna Fix For: 3.0.x Team, i have a table , which has a column defined as set , now i want to calculate the size of that column , if want see if i have any null values for that column . please let me know if we have any way to find out all the null values for that column . Many thanks in advance . Regards , RK. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14568) Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages
[ https://issues.apache.org/jira/browse/CASSANDRA-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545537#comment-16545537 ] Benedict commented on CASSANDRA-14568: -- Committed as d52c7b8c59, 31d5d870f9 and d3a994b105 tests were run against [circleci|https://circleci.com/workflow-run/c818c58a-83e3-4731-89d7-ab0b04d26d62] (unfortunately right now this link is down); this showed some failing auth dtests, but these code paths were unaffected by this patch and I confirmed they pass locally. > Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages > > > Key: CASSANDRA-14568 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14568 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 3.0.x, 3.11.x > > > In 2.1 and 2.2, row and complex deletions were represented as range > tombstones. LegacyLayout is our compatibility layer, that translates the > relevant RT patterns in 2.1/2.2 to row/complex deletions in 3.0, and vice > versa. Unfortunately, it does not handle the special case of static row > deletions, they are treated as regular row deletions. Since static rows are > themselves never directly deleted, the only issue is with collection > deletions. > Collection deletions in 2.1/2.2 were encoded as a range tombstone, consisting > of a sequence of the clustering keys’ data for the affected row, followed by > the bytes representing the name of the collection column. STATIC_CLUSTERING > contains zero clusterings, so by treating the deletion as for a regular row, > zero clusterings are written to precede the column name of the erased > collection, so the column name is written at position zero. > This can exhibit itself in at least two ways: > # If the type of your first clustering key is a variable width type, new > deletes will begin appearing covering the clustering key represented by the > column name. > ** If you have multiple clustering keys, you will receive a RT covering all > those rows with a matching first clustering key. > ** This RT will be valid as far as the system is concerned, and go > undetected unless there are outside data quality checks in place. > # Otherwise, an invalid size of data will be written to the clustering and > sent over the network to the 2.1 node. > ** The 2.1/2.2 node will handle this just fine, even though the record is > junk. Since it is a deletion covering impossible data, there will be no > user-API visible effect. But if received as a write from a 3.0 node, it will > dutifully persist the junk record. > ** The 3.0 node that originally sent this junk, may later coordinate a read > of the partition, and will notice a digest mismatch, read-repair and > serialize the junk to disk > ** The sstable containing this record is now corrupt; the deserialization > expects fixed-width data, but it encounters too many (or too few) bytes, and > is now at an incorrect position to read its structural information > ** (Alternatively when the 2.1 node is upgraded this will occur on eventual > compaction) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14556) Optimize streaming path in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545510#comment-16545510 ] Ariel Weisberg commented on CASSANDRA-14556: I am +1 on this. I'll commit it tomorrow unless someone else wants to give it another look. FYI the dtest is [here|https://github.com/apache/cassandra-dtest/compare/master...dineshjoshi:faster-streaming-rev2] > Optimize streaming path in Cassandra > > > Key: CASSANDRA-14556 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14556 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Major > Labels: Performance > Fix For: 4.x > > > During streaming, Cassandra reifies the sstables into objects. This creates > unnecessary garbage and slows down the whole streaming process as some > sstables can be transferred as a whole file rather than individual > partitions. The objective of the ticket is to detect when a whole sstable can > be transferred and skip the object reification. We can also use a zero-copy > path to avoid bringing data into user-space on both sending and receiving > side. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14556) Optimize streaming path in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14556: --- Status: Ready to Commit (was: Patch Available) > Optimize streaming path in Cassandra > > > Key: CASSANDRA-14556 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14556 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Major > Labels: Performance > Fix For: 4.x > > > During streaming, Cassandra reifies the sstables into objects. This creates > unnecessary garbage and slows down the whole streaming process as some > sstables can be transferred as a whole file rather than individual > partitions. The objective of the ticket is to detect when a whole sstable can > be transferred and skip the object reification. We can also use a zero-copy > path to avoid bringing data into user-space on both sending and receiving > side. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14565) NoHostAvailableException while executing create table statement
[ https://issues.apache.org/jira/browse/CASSANDRA-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545500#comment-16545500 ] Sumanth Pasupuleti commented on CASSANDRA-14565: [~kunal.deshpa...@honeywell.com] a few questions to understand the context * What consistency level (CL) do you use for read/write? And what CL do you use for create statement? * How many nodes does your cluster comprise of? * What version of C* are you using? * I assume this is CQL, and not thrift (based on the components you selected) - is this correct? If you can share the stack trace from the log, that would be great > NoHostAvailableException while executing create table statement > --- > > Key: CASSANDRA-14565 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14565 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Kunal Deshpande >Priority: Major > > Hi Team, > > We are facing very strange issue with single node cassandra database instance. > The Database is allowing Data read or write, but while creating new column > family\table we are getting > h1. NoHostAvailableException > > Any help in this regard will be appreciated. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[1/6] cassandra git commit: Fix corrupted static collection deletions in 3.0 -> 2.{1, 2} messages
Repository: cassandra Updated Branches: refs/heads/cassandra-3.0 6ce887e58 -> d52c7b8c5 refs/heads/cassandra-3.11 6bcc60ae4 -> 31d5d870f refs/heads/trunk 00b3edaa3 -> d3a994b10 Fix corrupted static collection deletions in 3.0 -> 2.{1,2} messages patch by Benedict; reviewed by Aleksey Yeschenko for CASSANDRA-14568 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d52c7b8c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d52c7b8c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d52c7b8c Branch: refs/heads/cassandra-3.0 Commit: d52c7b8c595cc0d06fc3607bf16e3f595f016bb6 Parents: 6ce887e Author: Benedict Elliott Smith Authored: Mon Jul 16 17:37:24 2018 +0100 Committer: Benedict Elliott Smith Committed: Mon Jul 16 17:40:09 2018 +0100 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/LegacyLayout.java | 15 ++- .../cassandra/db/marshal/CompositeType.java | 13 ++- .../db/partitions/PartitionUpdate.java | 40 +-- .../apache/cassandra/db/LegacyLayoutTest.java | 112 ++- 5 files changed, 155 insertions(+), 26 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d52c7b8c/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 6f6ae20..47e5cd0 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.17 + * Fix corrupted static collection deletions in 3.0 -> 2.{1,2} messages (CASSANDRA-14568) * Fix potential IndexOutOfBoundsException with counters (CASSANDRA-14167) * Restore resumable hints delivery, backport CASSANDRA-11960 (CASSANDRA-14419) * Always close RT markers returned by ReadCommand#executeLocally() (CASSANDRA-14515) http://git-wip-us.apache.org/repos/asf/cassandra/blob/d52c7b8c/src/java/org/apache/cassandra/db/LegacyLayout.java -- diff --git a/src/java/org/apache/cassandra/db/LegacyLayout.java b/src/java/org/apache/cassandra/db/LegacyLayout.java index 912d591..e0f66e3 100644 --- a/src/java/org/apache/cassandra/db/LegacyLayout.java +++ b/src/java/org/apache/cassandra/db/LegacyLayout.java @@ -197,6 +197,7 @@ public abstract class LegacyLayout int clusteringSize = metadata.comparator.size(); +boolean isStatic = metadata.isCompound() && CompositeType.isStaticName(bound); List components = CompositeType.splitName(bound); byte eoc = CompositeType.lastEOC(bound); @@ -206,8 +207,12 @@ public abstract class LegacyLayout assert components.size() <= clusteringSize || (!metadata.isCompactTable() && components.size() == clusteringSize + 1); ColumnDefinition collectionName = null; -if (components.size() > clusteringSize) -collectionName = metadata.getColumnDefinition(components.remove(clusteringSize)); +if (components.size() > (isStatic ? 0 : clusteringSize)) +{ +// pop the collection name from the back of the list of clusterings +ByteBuffer collectionNameBytes = components.remove(isStatic ? 0 : clusteringSize); +collectionName = metadata.getColumnDefinition(collectionNameBytes); +} boolean isInclusive; if (isStart) @@ -231,7 +236,7 @@ public abstract class LegacyLayout Slice.Bound.Kind boundKind = Slice.Bound.boundKind(isStart, isInclusive); Slice.Bound sb = Slice.Bound.create(boundKind, components.toArray(new ByteBuffer[components.size()])); -return new LegacyBound(sb, metadata.isCompound() && CompositeType.isStaticName(bound), collectionName); +return new LegacyBound(sb, isStatic, collectionName); } public static ByteBuffer encodeBound(CFMetaData metadata, Slice.Bound bound, boolean isStart) @@ -2386,8 +2391,8 @@ public abstract class LegacyLayout LegacyBound start = starts[i]; LegacyBound end = ends[i]; -CompositeType.Builder startBuilder = type.builder(); -CompositeType.Builder endBuilder = type.builder(); +CompositeType.Builder startBuilder = type.builder(start.isStatic); +CompositeType.Builder endBuilder = type.builder(end.isStatic); for (int j = 0; j < start.bound.clustering().size(); j++) { startBuilder.add(start.bound.get(j)); http://git-wip-us.apache.org/repos/asf/cassandra/blob/d52c7b8c/src/java/org/apache/cassandra/db/marshal/CompositeType.java -- diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
[3/6] cassandra git commit: Fix corrupted static collection deletions in 3.0 -> 2.{1, 2} messages
Fix corrupted static collection deletions in 3.0 -> 2.{1,2} messages patch by Benedict; reviewed by Aleksey Yeschenko for CASSANDRA-14568 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d52c7b8c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d52c7b8c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d52c7b8c Branch: refs/heads/trunk Commit: d52c7b8c595cc0d06fc3607bf16e3f595f016bb6 Parents: 6ce887e Author: Benedict Elliott Smith Authored: Mon Jul 16 17:37:24 2018 +0100 Committer: Benedict Elliott Smith Committed: Mon Jul 16 17:40:09 2018 +0100 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/LegacyLayout.java | 15 ++- .../cassandra/db/marshal/CompositeType.java | 13 ++- .../db/partitions/PartitionUpdate.java | 40 +-- .../apache/cassandra/db/LegacyLayoutTest.java | 112 ++- 5 files changed, 155 insertions(+), 26 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d52c7b8c/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 6f6ae20..47e5cd0 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.17 + * Fix corrupted static collection deletions in 3.0 -> 2.{1,2} messages (CASSANDRA-14568) * Fix potential IndexOutOfBoundsException with counters (CASSANDRA-14167) * Restore resumable hints delivery, backport CASSANDRA-11960 (CASSANDRA-14419) * Always close RT markers returned by ReadCommand#executeLocally() (CASSANDRA-14515) http://git-wip-us.apache.org/repos/asf/cassandra/blob/d52c7b8c/src/java/org/apache/cassandra/db/LegacyLayout.java -- diff --git a/src/java/org/apache/cassandra/db/LegacyLayout.java b/src/java/org/apache/cassandra/db/LegacyLayout.java index 912d591..e0f66e3 100644 --- a/src/java/org/apache/cassandra/db/LegacyLayout.java +++ b/src/java/org/apache/cassandra/db/LegacyLayout.java @@ -197,6 +197,7 @@ public abstract class LegacyLayout int clusteringSize = metadata.comparator.size(); +boolean isStatic = metadata.isCompound() && CompositeType.isStaticName(bound); List components = CompositeType.splitName(bound); byte eoc = CompositeType.lastEOC(bound); @@ -206,8 +207,12 @@ public abstract class LegacyLayout assert components.size() <= clusteringSize || (!metadata.isCompactTable() && components.size() == clusteringSize + 1); ColumnDefinition collectionName = null; -if (components.size() > clusteringSize) -collectionName = metadata.getColumnDefinition(components.remove(clusteringSize)); +if (components.size() > (isStatic ? 0 : clusteringSize)) +{ +// pop the collection name from the back of the list of clusterings +ByteBuffer collectionNameBytes = components.remove(isStatic ? 0 : clusteringSize); +collectionName = metadata.getColumnDefinition(collectionNameBytes); +} boolean isInclusive; if (isStart) @@ -231,7 +236,7 @@ public abstract class LegacyLayout Slice.Bound.Kind boundKind = Slice.Bound.boundKind(isStart, isInclusive); Slice.Bound sb = Slice.Bound.create(boundKind, components.toArray(new ByteBuffer[components.size()])); -return new LegacyBound(sb, metadata.isCompound() && CompositeType.isStaticName(bound), collectionName); +return new LegacyBound(sb, isStatic, collectionName); } public static ByteBuffer encodeBound(CFMetaData metadata, Slice.Bound bound, boolean isStart) @@ -2386,8 +2391,8 @@ public abstract class LegacyLayout LegacyBound start = starts[i]; LegacyBound end = ends[i]; -CompositeType.Builder startBuilder = type.builder(); -CompositeType.Builder endBuilder = type.builder(); +CompositeType.Builder startBuilder = type.builder(start.isStatic); +CompositeType.Builder endBuilder = type.builder(end.isStatic); for (int j = 0; j < start.bound.clustering().size(); j++) { startBuilder.add(start.bound.get(j)); http://git-wip-us.apache.org/repos/asf/cassandra/blob/d52c7b8c/src/java/org/apache/cassandra/db/marshal/CompositeType.java -- diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java b/src/java/org/apache/cassandra/db/marshal/CompositeType.java index 52d6d39..d4ddfc0 100644 --- a/src/java/org/apache/cassandra/db/marshal/CompositeType.java +++ b/src/java/org/apache/cassandra/db/marshal/CompositeType.java @@ -339,6 +339,11 @@ public
[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d3a994b1 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d3a994b1 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d3a994b1 Branch: refs/heads/trunk Commit: d3a994b10537809267ec5ebae784ee5497cdb265 Parents: 00b3eda 31d5d87 Author: Benedict Elliott Smith Authored: Mon Jul 16 17:50:44 2018 +0100 Committer: Benedict Elliott Smith Committed: Mon Jul 16 17:50:44 2018 +0100 -- CHANGES.txt | 1 + .../db/partitions/PartitionUpdate.java | 40 ++-- 2 files changed, 29 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d3a994b1/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d3a994b1/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java -- diff --cc src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java index 2dc566a,4ca8f1c..b793420 --- a/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java +++ b/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java @@@ -122,19 -162,34 +123,34 @@@ public class PartitionUpdate extends Ab * * @return the newly created partition update containing only {@code row}. */ - public static PartitionUpdate singleRowUpdate(TableMetadata metadata, DecoratedKey key, Row row) -public static PartitionUpdate singleRowUpdate(CFMetaData metadata, DecoratedKey key, Row row, Row staticRow) ++public static PartitionUpdate singleRowUpdate(TableMetadata metadata, DecoratedKey key, Row row, Row staticRow) { MutableDeletionInfo deletionInfo = MutableDeletionInfo.live(); - if (row.isStatic()) - { - Holder holder = new Holder(new RegularAndStaticColumns(Columns.from(row.columns()), Columns.NONE), BTree.empty(), deletionInfo, row, EncodingStats.NO_STATS); - return new PartitionUpdate(metadata, key, holder, deletionInfo, false); - } - else - { - Holder holder = new Holder(new RegularAndStaticColumns(Columns.NONE, Columns.from(row.columns())), BTree.singleton(row), deletionInfo, Rows.EMPTY_STATIC_ROW, EncodingStats.NO_STATS); - return new PartitionUpdate(metadata, key, holder, deletionInfo, false); - } + Holder holder = new Holder( -new PartitionColumns( ++new RegularAndStaticColumns( + staticRow == null ? Columns.NONE : Columns.from(staticRow.columns()), + row == null ? Columns.NONE : Columns.from(row.columns()) + ), + row == null ? BTree.empty() : BTree.singleton(row), + deletionInfo, + staticRow == null ? Rows.EMPTY_STATIC_ROW : staticRow, + EncodingStats.NO_STATS + ); + return new PartitionUpdate(metadata, key, holder, deletionInfo, false); + } + + /** + * Creates an immutable partition update that contains a single row update. + * + * @param metadata the metadata for the created update. + * @param key the partition key for the partition to update. + * @param row the row for the update (may be static). + * + * @return the newly created partition update containing only {@code row}. + */ -public static PartitionUpdate singleRowUpdate(CFMetaData metadata, DecoratedKey key, Row row) ++public static PartitionUpdate singleRowUpdate(TableMetadata metadata, DecoratedKey key, Row row) + { + return singleRowUpdate(metadata, key, row.isStatic() ? null : row, row.isStatic() ? row : null); } /** - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into HEAD
Merge branch 'cassandra-3.0' into HEAD Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/31d5d870 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/31d5d870 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/31d5d870 Branch: refs/heads/trunk Commit: 31d5d870f9f5b56391db46ba6cdf9e0882d8a5c0 Parents: 6bcc60a d52c7b8 Author: Benedict Elliott Smith Authored: Mon Jul 16 17:46:37 2018 +0100 Committer: Benedict Elliott Smith Committed: Mon Jul 16 17:46:37 2018 +0100 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/LegacyLayout.java | 15 ++- .../cassandra/db/marshal/CompositeType.java | 13 ++- .../db/partitions/PartitionUpdate.java | 40 +--- .../apache/cassandra/db/LegacyLayoutTest.java | 99 +++- 5 files changed, 145 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/31d5d870/CHANGES.txt -- diff --cc CHANGES.txt index 5beccfb,47e5cd0..5e39b5e --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,24 -1,7 +1,25 @@@ -3.0.17 +3.11.3 + * Validate supported column type with SASI analyzer (CASSANDRA-13669) + * Remove BTree.Builder Recycler to reduce memory usage (CASSANDRA-13929) + * Reduce nodetool GC thread count (CASSANDRA-14475) + * Fix New SASI view creation during Index Redistribution (CASSANDRA-14055) + * Remove string formatting lines from BufferPool hot path (CASSANDRA-14416) + * Update metrics to 3.1.5 (CASSANDRA-12924) + * Detect OpenJDK jvm type and architecture (CASSANDRA-12793) + * Don't use guava collections in the non-system keyspace jmx attributes (CASSANDRA-12271) + * Allow existing nodes to use all peers in shadow round (CASSANDRA-13851) + * Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299) + * Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370) + * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891) + * Serialize empty buffer as empty string for json output format (CASSANDRA-14245) + * Allow logging implementation to be interchanged for embedded testing (CASSANDRA-13396) + * SASI tokenizer for simple delimiter based entries (CASSANDRA-14247) + * Fix Loss of digits when doing CAST from varint/bigint to decimal (CASSANDRA-14170) + * RateBasedBackPressure unnecessarily invokes a lock on the Guava RateLimiter (CASSANDRA-14163) + * Fix wildcard GROUP BY queries (CASSANDRA-14209) +Merged from 3.0: + * Fix corrupted static collection deletions in 3.0 -> 2.{1,2} messages (CASSANDRA-14568) * Fix potential IndexOutOfBoundsException with counters (CASSANDRA-14167) - * Restore resumable hints delivery, backport CASSANDRA-11960 (CASSANDRA-14419) * Always close RT markers returned by ReadCommand#executeLocally() (CASSANDRA-14515) * Reverse order queries with range tombstones can cause data loss (CASSANDRA-14513) * Fix regression of lagging commitlog flush log message (CASSANDRA-14451) http://git-wip-us.apache.org/repos/asf/cassandra/blob/31d5d870/src/java/org/apache/cassandra/db/LegacyLayout.java -- diff --cc src/java/org/apache/cassandra/db/LegacyLayout.java index 1cfe622,e0f66e3..eed4113 --- a/src/java/org/apache/cassandra/db/LegacyLayout.java +++ b/src/java/org/apache/cassandra/db/LegacyLayout.java @@@ -229,14 -234,14 +234,14 @@@ public abstract class LegacyLayou } } -Slice.Bound.Kind boundKind = Slice.Bound.boundKind(isStart, isInclusive); -Slice.Bound sb = Slice.Bound.create(boundKind, components.toArray(new ByteBuffer[components.size()])); -return new LegacyBound(sb, isStatic, collectionName); +ClusteringPrefix.Kind boundKind = ClusteringBound.boundKind(isStart, isInclusive); +ClusteringBound cb = ClusteringBound.create(boundKind, components.toArray(new ByteBuffer[components.size()])); - return new LegacyBound(cb, metadata.isCompound() && CompositeType.isStaticName(bound), collectionName); ++return new LegacyBound(cb, isStatic, collectionName); } -public static ByteBuffer encodeBound(CFMetaData metadata, Slice.Bound bound, boolean isStart) +public static ByteBuffer encodeBound(CFMetaData metadata, ClusteringBound bound, boolean isStart) { -if (bound == Slice.Bound.BOTTOM || bound == Slice.Bound.TOP || metadata.comparator.size() == 0) +if (bound == ClusteringBound.BOTTOM || bound == ClusteringBound.TOP || metadata.comparator.size() == 0) return ByteBufferUtil.EMPTY_BYTE_BUFFER; ClusteringPrefix clustering = bound.clustering();
[2/6] cassandra git commit: Fix corrupted static collection deletions in 3.0 -> 2.{1, 2} messages
Fix corrupted static collection deletions in 3.0 -> 2.{1,2} messages patch by Benedict; reviewed by Aleksey Yeschenko for CASSANDRA-14568 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d52c7b8c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d52c7b8c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d52c7b8c Branch: refs/heads/cassandra-3.11 Commit: d52c7b8c595cc0d06fc3607bf16e3f595f016bb6 Parents: 6ce887e Author: Benedict Elliott Smith Authored: Mon Jul 16 17:37:24 2018 +0100 Committer: Benedict Elliott Smith Committed: Mon Jul 16 17:40:09 2018 +0100 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/LegacyLayout.java | 15 ++- .../cassandra/db/marshal/CompositeType.java | 13 ++- .../db/partitions/PartitionUpdate.java | 40 +-- .../apache/cassandra/db/LegacyLayoutTest.java | 112 ++- 5 files changed, 155 insertions(+), 26 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d52c7b8c/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 6f6ae20..47e5cd0 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.17 + * Fix corrupted static collection deletions in 3.0 -> 2.{1,2} messages (CASSANDRA-14568) * Fix potential IndexOutOfBoundsException with counters (CASSANDRA-14167) * Restore resumable hints delivery, backport CASSANDRA-11960 (CASSANDRA-14419) * Always close RT markers returned by ReadCommand#executeLocally() (CASSANDRA-14515) http://git-wip-us.apache.org/repos/asf/cassandra/blob/d52c7b8c/src/java/org/apache/cassandra/db/LegacyLayout.java -- diff --git a/src/java/org/apache/cassandra/db/LegacyLayout.java b/src/java/org/apache/cassandra/db/LegacyLayout.java index 912d591..e0f66e3 100644 --- a/src/java/org/apache/cassandra/db/LegacyLayout.java +++ b/src/java/org/apache/cassandra/db/LegacyLayout.java @@ -197,6 +197,7 @@ public abstract class LegacyLayout int clusteringSize = metadata.comparator.size(); +boolean isStatic = metadata.isCompound() && CompositeType.isStaticName(bound); List components = CompositeType.splitName(bound); byte eoc = CompositeType.lastEOC(bound); @@ -206,8 +207,12 @@ public abstract class LegacyLayout assert components.size() <= clusteringSize || (!metadata.isCompactTable() && components.size() == clusteringSize + 1); ColumnDefinition collectionName = null; -if (components.size() > clusteringSize) -collectionName = metadata.getColumnDefinition(components.remove(clusteringSize)); +if (components.size() > (isStatic ? 0 : clusteringSize)) +{ +// pop the collection name from the back of the list of clusterings +ByteBuffer collectionNameBytes = components.remove(isStatic ? 0 : clusteringSize); +collectionName = metadata.getColumnDefinition(collectionNameBytes); +} boolean isInclusive; if (isStart) @@ -231,7 +236,7 @@ public abstract class LegacyLayout Slice.Bound.Kind boundKind = Slice.Bound.boundKind(isStart, isInclusive); Slice.Bound sb = Slice.Bound.create(boundKind, components.toArray(new ByteBuffer[components.size()])); -return new LegacyBound(sb, metadata.isCompound() && CompositeType.isStaticName(bound), collectionName); +return new LegacyBound(sb, isStatic, collectionName); } public static ByteBuffer encodeBound(CFMetaData metadata, Slice.Bound bound, boolean isStart) @@ -2386,8 +2391,8 @@ public abstract class LegacyLayout LegacyBound start = starts[i]; LegacyBound end = ends[i]; -CompositeType.Builder startBuilder = type.builder(); -CompositeType.Builder endBuilder = type.builder(); +CompositeType.Builder startBuilder = type.builder(start.isStatic); +CompositeType.Builder endBuilder = type.builder(end.isStatic); for (int j = 0; j < start.bound.clustering().size(); j++) { startBuilder.add(start.bound.get(j)); http://git-wip-us.apache.org/repos/asf/cassandra/blob/d52c7b8c/src/java/org/apache/cassandra/db/marshal/CompositeType.java -- diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java b/src/java/org/apache/cassandra/db/marshal/CompositeType.java index 52d6d39..d4ddfc0 100644 --- a/src/java/org/apache/cassandra/db/marshal/CompositeType.java +++ b/src/java/org/apache/cassandra/db/marshal/CompositeType.java @@ -339,6 +339,11 @@
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into HEAD
Merge branch 'cassandra-3.0' into HEAD Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/31d5d870 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/31d5d870 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/31d5d870 Branch: refs/heads/cassandra-3.11 Commit: 31d5d870f9f5b56391db46ba6cdf9e0882d8a5c0 Parents: 6bcc60a d52c7b8 Author: Benedict Elliott Smith Authored: Mon Jul 16 17:46:37 2018 +0100 Committer: Benedict Elliott Smith Committed: Mon Jul 16 17:46:37 2018 +0100 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/LegacyLayout.java | 15 ++- .../cassandra/db/marshal/CompositeType.java | 13 ++- .../db/partitions/PartitionUpdate.java | 40 +--- .../apache/cassandra/db/LegacyLayoutTest.java | 99 +++- 5 files changed, 145 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/31d5d870/CHANGES.txt -- diff --cc CHANGES.txt index 5beccfb,47e5cd0..5e39b5e --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,24 -1,7 +1,25 @@@ -3.0.17 +3.11.3 + * Validate supported column type with SASI analyzer (CASSANDRA-13669) + * Remove BTree.Builder Recycler to reduce memory usage (CASSANDRA-13929) + * Reduce nodetool GC thread count (CASSANDRA-14475) + * Fix New SASI view creation during Index Redistribution (CASSANDRA-14055) + * Remove string formatting lines from BufferPool hot path (CASSANDRA-14416) + * Update metrics to 3.1.5 (CASSANDRA-12924) + * Detect OpenJDK jvm type and architecture (CASSANDRA-12793) + * Don't use guava collections in the non-system keyspace jmx attributes (CASSANDRA-12271) + * Allow existing nodes to use all peers in shadow round (CASSANDRA-13851) + * Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299) + * Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370) + * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891) + * Serialize empty buffer as empty string for json output format (CASSANDRA-14245) + * Allow logging implementation to be interchanged for embedded testing (CASSANDRA-13396) + * SASI tokenizer for simple delimiter based entries (CASSANDRA-14247) + * Fix Loss of digits when doing CAST from varint/bigint to decimal (CASSANDRA-14170) + * RateBasedBackPressure unnecessarily invokes a lock on the Guava RateLimiter (CASSANDRA-14163) + * Fix wildcard GROUP BY queries (CASSANDRA-14209) +Merged from 3.0: + * Fix corrupted static collection deletions in 3.0 -> 2.{1,2} messages (CASSANDRA-14568) * Fix potential IndexOutOfBoundsException with counters (CASSANDRA-14167) - * Restore resumable hints delivery, backport CASSANDRA-11960 (CASSANDRA-14419) * Always close RT markers returned by ReadCommand#executeLocally() (CASSANDRA-14515) * Reverse order queries with range tombstones can cause data loss (CASSANDRA-14513) * Fix regression of lagging commitlog flush log message (CASSANDRA-14451) http://git-wip-us.apache.org/repos/asf/cassandra/blob/31d5d870/src/java/org/apache/cassandra/db/LegacyLayout.java -- diff --cc src/java/org/apache/cassandra/db/LegacyLayout.java index 1cfe622,e0f66e3..eed4113 --- a/src/java/org/apache/cassandra/db/LegacyLayout.java +++ b/src/java/org/apache/cassandra/db/LegacyLayout.java @@@ -229,14 -234,14 +234,14 @@@ public abstract class LegacyLayou } } -Slice.Bound.Kind boundKind = Slice.Bound.boundKind(isStart, isInclusive); -Slice.Bound sb = Slice.Bound.create(boundKind, components.toArray(new ByteBuffer[components.size()])); -return new LegacyBound(sb, isStatic, collectionName); +ClusteringPrefix.Kind boundKind = ClusteringBound.boundKind(isStart, isInclusive); +ClusteringBound cb = ClusteringBound.create(boundKind, components.toArray(new ByteBuffer[components.size()])); - return new LegacyBound(cb, metadata.isCompound() && CompositeType.isStaticName(bound), collectionName); ++return new LegacyBound(cb, isStatic, collectionName); } -public static ByteBuffer encodeBound(CFMetaData metadata, Slice.Bound bound, boolean isStart) +public static ByteBuffer encodeBound(CFMetaData metadata, ClusteringBound bound, boolean isStart) { -if (bound == Slice.Bound.BOTTOM || bound == Slice.Bound.TOP || metadata.comparator.size() == 0) +if (bound == ClusteringBound.BOTTOM || bound == ClusteringBound.TOP || metadata.comparator.size() == 0) return ByteBufferUtil.EMPTY_BYTE_BUFFER; ClusteringPrefix clustering = bound.clustering();
[jira] [Commented] (CASSANDRA-14556) Optimize streaming path in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545474#comment-16545474 ] ASF GitHub Bot commented on CASSANDRA-14556: Github user aweisberg commented on the issue: https://github.com/apache/cassandra/pull/239 Where is the dtest? The existing link doesn't seem to work and when I looked at your repo there are no branches with these tests? > Optimize streaming path in Cassandra > > > Key: CASSANDRA-14556 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14556 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Major > Labels: Performance > Fix For: 4.x > > > During streaming, Cassandra reifies the sstables into objects. This creates > unnecessary garbage and slows down the whole streaming process as some > sstables can be transferred as a whole file rather than individual > partitions. The objective of the ticket is to detect when a whole sstable can > be transferred and skip the object reification. We can also use a zero-copy > path to avoid bringing data into user-space on both sending and receiving > side. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r1836051 - in /cassandra/site: publish/community/index.html src/community.md
Author: mshuler Date: Mon Jul 16 15:48:24 2018 New Revision: 1836051 URL: http://svn.apache.org/viewvc?rev=1836051=rev Log: Drop Planet Cassandra link Modified: cassandra/site/publish/community/index.html cassandra/site/src/community.md Modified: cassandra/site/publish/community/index.html URL: http://svn.apache.org/viewvc/cassandra/site/publish/community/index.html?rev=1836051=1836050=1836051=diff == --- cassandra/site/publish/community/index.html (original) +++ cassandra/site/publish/community/index.html Mon Jul 16 15:48:24 2018 @@ -131,11 +131,6 @@ mostly useful for Cassandra developers a You can also check the http://stackoverflow.com/questions/tagged/cassandra;>QA about using Cassandra on Stack Overflow. -News, articles and user groups - -You can find a number of news, articles and use cases, as well a links to Cassandra user groups on the http://planetcassandra.org/;>Planet -Cassandra (not endorsed by Apache). - Books and Publications Modified: cassandra/site/src/community.md URL: http://svn.apache.org/viewvc/cassandra/site/src/community.md?rev=1836051=1836050=1836051=diff == --- cassandra/site/src/community.md (original) +++ cassandra/site/src/community.md Mon Jul 16 15:48:24 2018 @@ -40,11 +40,6 @@ You can also check the [Q about using Overflow. -### News, articles and user groups - -You can find a number of news, articles and use cases, as well a links to Cassandra user groups on the [Planet -Cassandra](http://planetcassandra.org/) (not endorsed by Apache). - ### Books and Publications * [Cassandra: The Definitive Guide, 2nd Edition](http://shop.oreilly.com/product/0636920043041.do), by Jeff Carpenter and Eben Hewitt. Updated for Cassandra 3.0 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14567) CQL query returns different results in 2.2.5 and 3.0.15
[ https://issues.apache.org/jira/browse/CASSANDRA-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545307#comment-16545307 ] Benedict commented on CASSANDRA-14567: -- This is a little mind melting. But it seems like whatever semantics we pick need to be consistent across all possible clusterings - for both least surprise/confusion for developers, and so that MVs/2i's provide consistent results. AFAICT the 3.0 semantics provide this, and are intuitive, whereas the 2.2 do not, and are not. > CQL query returns different results in 2.2.5 and 3.0.15 > --- > > Key: CASSANDRA-14567 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14567 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Dikang Gu >Priority: Major > > During our 2.2.5 to 3.0.15 upgrade, we find the a cql query returns different > results in 2.2.5 and 3.0.15, here is a unit test to reproduce it, > [https://gist.github.com/DikangGu/e538ed2de22b74e49b8dd43f7093a996] > > In C* 2.2.5, it returns all 4 rows: *[Row[1, 10, 0, 1], Row[1, 10, 10, 2], > Row[1, 1, 0, 3], Row[1, 1, 10, 4]]* > While in C* 3.0.15, it only returns 2 rows to client: *[Row[1, 10, 0, 1], > Row[1, 1, 10, 4]]* > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14568) Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages
[ https://issues.apache.org/jira/browse/CASSANDRA-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14568: -- Fix Version/s: (was: 3.0.18) 3.11.x 3.0.x Status: Patch Available (was: Open) > Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages > > > Key: CASSANDRA-14568 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14568 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 3.0.x, 3.11.x > > > In 2.1 and 2.2, row and complex deletions were represented as range > tombstones. LegacyLayout is our compatibility layer, that translates the > relevant RT patterns in 2.1/2.2 to row/complex deletions in 3.0, and vice > versa. Unfortunately, it does not handle the special case of static row > deletions, they are treated as regular row deletions. Since static rows are > themselves never directly deleted, the only issue is with collection > deletions. > Collection deletions in 2.1/2.2 were encoded as a range tombstone, consisting > of a sequence of the clustering keys’ data for the affected row, followed by > the bytes representing the name of the collection column. STATIC_CLUSTERING > contains zero clusterings, so by treating the deletion as for a regular row, > zero clusterings are written to precede the column name of the erased > collection, so the column name is written at position zero. > This can exhibit itself in at least two ways: > # If the type of your first clustering key is a variable width type, new > deletes will begin appearing covering the clustering key represented by the > column name. > ** If you have multiple clustering keys, you will receive a RT covering all > those rows with a matching first clustering key. > ** This RT will be valid as far as the system is concerned, and go > undetected unless there are outside data quality checks in place. > # Otherwise, an invalid size of data will be written to the clustering and > sent over the network to the 2.1 node. > ** The 2.1/2.2 node will handle this just fine, even though the record is > junk. Since it is a deletion covering impossible data, there will be no > user-API visible effect. But if received as a write from a 3.0 node, it will > dutifully persist the junk record. > ** The 3.0 node that originally sent this junk, may later coordinate a read > of the partition, and will notice a digest mismatch, read-repair and > serialize the junk to disk > ** The sstable containing this record is now corrupt; the deserialization > expects fixed-width data, but it encounters too many (or too few) bytes, and > is now at an incorrect position to read its structural information > ** (Alternatively when the 2.1 node is upgraded this will occur on eventual > compaction) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14568) Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages
[ https://issues.apache.org/jira/browse/CASSANDRA-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14568: -- Status: Ready to Commit (was: Patch Available) > Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages > > > Key: CASSANDRA-14568 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14568 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 3.0.x, 3.11.x > > > In 2.1 and 2.2, row and complex deletions were represented as range > tombstones. LegacyLayout is our compatibility layer, that translates the > relevant RT patterns in 2.1/2.2 to row/complex deletions in 3.0, and vice > versa. Unfortunately, it does not handle the special case of static row > deletions, they are treated as regular row deletions. Since static rows are > themselves never directly deleted, the only issue is with collection > deletions. > Collection deletions in 2.1/2.2 were encoded as a range tombstone, consisting > of a sequence of the clustering keys’ data for the affected row, followed by > the bytes representing the name of the collection column. STATIC_CLUSTERING > contains zero clusterings, so by treating the deletion as for a regular row, > zero clusterings are written to precede the column name of the erased > collection, so the column name is written at position zero. > This can exhibit itself in at least two ways: > # If the type of your first clustering key is a variable width type, new > deletes will begin appearing covering the clustering key represented by the > column name. > ** If you have multiple clustering keys, you will receive a RT covering all > those rows with a matching first clustering key. > ** This RT will be valid as far as the system is concerned, and go > undetected unless there are outside data quality checks in place. > # Otherwise, an invalid size of data will be written to the clustering and > sent over the network to the 2.1 node. > ** The 2.1/2.2 node will handle this just fine, even though the record is > junk. Since it is a deletion covering impossible data, there will be no > user-API visible effect. But if received as a write from a 3.0 node, it will > dutifully persist the junk record. > ** The 3.0 node that originally sent this junk, may later coordinate a read > of the partition, and will notice a digest mismatch, read-repair and > serialize the junk to disk > ** The sstable containing this record is now corrupt; the deserialization > expects fixed-width data, but it encounters too many (or too few) bytes, and > is now at an incorrect position to read its structural information > ** (Alternatively when the 2.1 node is upgraded this will occur on eventual > compaction) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14568) Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages
[ https://issues.apache.org/jira/browse/CASSANDRA-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545241#comment-16545241 ] Aleksey Yeschenko commented on CASSANDRA-14568: --- +1, ship it. > Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages > > > Key: CASSANDRA-14568 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14568 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 3.0.18 > > > In 2.1 and 2.2, row and complex deletions were represented as range > tombstones. LegacyLayout is our compatibility layer, that translates the > relevant RT patterns in 2.1/2.2 to row/complex deletions in 3.0, and vice > versa. Unfortunately, it does not handle the special case of static row > deletions, they are treated as regular row deletions. Since static rows are > themselves never directly deleted, the only issue is with collection > deletions. > Collection deletions in 2.1/2.2 were encoded as a range tombstone, consisting > of a sequence of the clustering keys’ data for the affected row, followed by > the bytes representing the name of the collection column. STATIC_CLUSTERING > contains zero clusterings, so by treating the deletion as for a regular row, > zero clusterings are written to precede the column name of the erased > collection, so the column name is written at position zero. > This can exhibit itself in at least two ways: > # If the type of your first clustering key is a variable width type, new > deletes will begin appearing covering the clustering key represented by the > column name. > ** If you have multiple clustering keys, you will receive a RT covering all > those rows with a matching first clustering key. > ** This RT will be valid as far as the system is concerned, and go > undetected unless there are outside data quality checks in place. > # Otherwise, an invalid size of data will be written to the clustering and > sent over the network to the 2.1 node. > ** The 2.1/2.2 node will handle this just fine, even though the record is > junk. Since it is a deletion covering impossible data, there will be no > user-API visible effect. But if received as a write from a 3.0 node, it will > dutifully persist the junk record. > ** The 3.0 node that originally sent this junk, may later coordinate a read > of the partition, and will notice a digest mismatch, read-repair and > serialize the junk to disk > ** The sstable containing this record is now corrupt; the deserialization > expects fixed-width data, but it encounters too many (or too few) bytes, and > is now at an incorrect position to read its structural information > ** (Alternatively when the 2.1 node is upgraded this will occur on eventual > compaction) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[Cassandra Wiki] Update of "Committers" by AlekseyYeschenko
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "Committers" page has been changed by AlekseyYeschenko: https://wiki.apache.org/cassandra/Committers?action=diff=77=78 ||Marcus Eriksson ||Apr 2013 ||Apple ||PMC member || ||Mikhail Stepura ||Jan 2014 ||Apple || || ||Tyler Hobbs ||Mar 2014 ||Datastax ||PMC member || - ||Benedict Elliott Smith ||May 2014 ||Vast || || + ||Benedict Elliott Smith ||May 2014 ||Apple || || ||Josh Mckenzie ||Jul 2014 ||Datastax ||PMC member || ||Robert Stupp ||Jan 2015 ||Datastax || || ||Sam Tunnicliffe ||May 2015 ||Apple || || - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14567) CQL query returns different results in 2.2.5 and 3.0.15
[ https://issues.apache.org/jira/browse/CASSANDRA-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545057#comment-16545057 ] Aleksey Yeschenko edited comment on CASSANDRA-14567 at 7/16/18 10:57 AM: - Interesting. I think 3.0 behaviour is correct here, and it’s 2.2 that’s broken? Am I missing something? EDIT: actually, the semantics of this are confusing, I don't even know what the right behaviour should be. was (Author: iamaleksey): Interesting. I think 3.0 behaviour is correct here, and it’s 2.2 that’s broken? Am I missing something? > CQL query returns different results in 2.2.5 and 3.0.15 > --- > > Key: CASSANDRA-14567 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14567 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Dikang Gu >Priority: Major > > During our 2.2.5 to 3.0.15 upgrade, we find the a cql query returns different > results in 2.2.5 and 3.0.15, here is a unit test to reproduce it, > [https://gist.github.com/DikangGu/e538ed2de22b74e49b8dd43f7093a996] > > In C* 2.2.5, it returns all 4 rows: *[Row[1, 10, 0, 1], Row[1, 10, 10, 2], > Row[1, 1, 0, 3], Row[1, 1, 10, 4]]* > While in C* 3.0.15, it only returns 2 rows to client: *[Row[1, 10, 0, 1], > Row[1, 1, 10, 4]]* > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14567) CQL query returns different results in 2.2.5 and 3.0.15
[ https://issues.apache.org/jira/browse/CASSANDRA-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545057#comment-16545057 ] Aleksey Yeschenko commented on CASSANDRA-14567: --- Interesting. I think 3.0 behaviour is correct here, and it’s 2.2 that’s broken? Am I missing something? > CQL query returns different results in 2.2.5 and 3.0.15 > --- > > Key: CASSANDRA-14567 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14567 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Dikang Gu >Priority: Major > > During our 2.2.5 to 3.0.15 upgrade, we find the a cql query returns different > results in 2.2.5 and 3.0.15, here is a unit test to reproduce it, > [https://gist.github.com/DikangGu/e538ed2de22b74e49b8dd43f7093a996] > > In C* 2.2.5, it returns all 4 rows: *[Row[1, 10, 0, 1], Row[1, 10, 10, 2], > Row[1, 1, 0, 3], Row[1, 1, 10, 4]]* > While in C* 3.0.15, it only returns 2 rows to client: *[Row[1, 10, 0, 1], > Row[1, 1, 10, 4]]* > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org