[jira] [Created] (CASSANDRA-14929) twcs sstables gets merged following node removal
Gil Ganz created CASSANDRA-14929:
Summary: twcs sstables gets merged following node removal
Key: CASSANDRA-14929
URL: https://issues.apache.org/jira/browse/CASSANDRA-14929
Project: Cassandra
Issue Type: Bug
Components: Compaction
Environment: cassandra 3.0.17
Reporter: Gil Ganz
after removing a node from the cluster, a table that is defined as twcs, has
sstables from different time windows merged together, making old and new data
sit in the same sstable.
CREATE KEYSPACE gil_test WITH replication = \{'class':
'NetworkTopologyStrategy', 'DC1': '2'} AND durable_writes = true;
CREATE TABLE gil_test.my_test (
id int,
creation_time timestamp,
name text,
PRIMARY KEY (id, creation_time)
) WITH CLUSTERING ORDER BY (creation_time ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = \{'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = \{'compaction_window_unit': 'HOURS',
'compaction_window_size': '2', 'class': 'TimeWindowCompactionStrategy'}
AND compression = \{'chunk_length_kb': '4', 'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 3600
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = 'NONE';
3 nodes cluster
before removing node number 3 - directory listing
drwxr-xr-x 2 cassandra cassandra 4096 Dec 10 20:28 backups
-rw-r--r-- 1 cassandra cassandra 51 Dec 10 22:10 mc-16-big-CompressionInfo.db
-rw-r--r-- 1 cassandra cassandra 2044 Dec 10 22:10 mc-16-big-Data.db
-rw-r--r-- 1 cassandra cassandra 9 Dec 10 22:10 mc-16-big-Digest.crc32
-rw-r--r-- 1 cassandra cassandra 64 Dec 10 22:10 mc-16-big-Filter.db
-rw-r--r-- 1 cassandra cassandra 375 Dec 10 22:10 mc-16-big-Index.db
-rw-r--r-- 1 cassandra cassandra 4805 Dec 10 22:10 mc-16-big-Statistics.db
-rw-r--r-- 1 cassandra cassandra 56 Dec 10 22:10 mc-16-big-Summary.db
-rw-r--r-- 1 cassandra cassandra 92 Dec 10 22:10 mc-16-big-TOC.txt
-rw-r--r-- 1 cassandra cassandra 51 Dec 11 00:00 mc-31-big-CompressionInfo.db
-rw-r--r-- 1 cassandra cassandra 2343 Dec 11 00:00 mc-31-big-Data.db
-rw-r--r-- 1 cassandra cassandra 10 Dec 11 00:00 mc-31-big-Digest.crc32
-rw-r--r-- 1 cassandra cassandra 72 Dec 11 00:00 mc-31-big-Filter.db
-rw-r--r-- 1 cassandra cassandra 411 Dec 11 00:00 mc-31-big-Index.db
-rw-r--r-- 1 cassandra cassandra 4816 Dec 11 00:00 mc-31-big-Statistics.db
-rw-r--r-- 1 cassandra cassandra 56 Dec 11 00:00 mc-31-big-Summary.db
-rw-r--r-- 1 cassandra cassandra 92 Dec 11 00:00 mc-31-big-TOC.txt
-rw-r--r-- 1 cassandra cassandra 51 Dec 11 02:00 mc-47-big-CompressionInfo.db
-rw-r--r-- 1 cassandra cassandra 2732 Dec 11 02:00 mc-47-big-Data.db
-rw-r--r-- 1 cassandra cassandra 9 Dec 11 02:00 mc-47-big-Digest.crc32
-rw-r--r-- 1 cassandra cassandra 80 Dec 11 02:00 mc-47-big-Filter.db
-rw-r--r-- 1 cassandra cassandra 475 Dec 11 02:00 mc-47-big-Index.db
-rw-r--r-- 1 cassandra cassandra 4835 Dec 11 02:00 mc-47-big-Statistics.db
-rw-r--r-- 1 cassandra cassandra 56 Dec 11 02:00 mc-47-big-Summary.db
-rw-r--r-- 1 cassandra cassandra 92 Dec 11 02:00 mc-47-big-TOC.txt
-rw-r--r-- 1 cassandra cassandra 51 Dec 11 04:00 mc-63-big-CompressionInfo.db
-rw-r--r-- 1 cassandra cassandra 2504 Dec 11 04:00 mc-63-big-Data.db
-rw-r--r-- 1 cassandra cassandra 10 Dec 11 04:00 mc-63-big-Digest.crc32
-rw-r--r-- 1 cassandra cassandra 72 Dec 11 04:00 mc-63-big-Filter.db
-rw-r--r-- 1 cassandra cassandra 430 Dec 11 04:00 mc-63-big-Index.db
-rw-r--r-- 1 cassandra cassandra 4823 Dec 11 04:00 mc-63-big-Statistics.db
-rw-r--r-- 1 cassandra cassandra 56 Dec 11 04:00 mc-63-big-Summary.db
-rw-r--r-- 1 cassandra cassandra 92 Dec 11 04:00 mc-63-big-TOC.txt
-rw-r--r-- 1 cassandra cassandra 43 Dec 11 06:10 mc-79-big-CompressionInfo.db
-rw-r--r-- 1 cassandra cassandra 317 Dec 11 06:10 mc-79-big-Data.db
-rw-r--r-- 1 cassandra cassandra 10 Dec 11 06:10 mc-79-big-Digest.crc32
-rw-r--r-- 1 cassandra cassandra 24 Dec 11 06:10 mc-79-big-Filter.db
-rw-r--r-- 1 cassandra cassandra 69 Dec 11 06:10 mc-79-big-Index.db
-rw-r--r-- 1 cassandra cassandra 4698 Dec 11 06:10 mc-79-big-Statistics.db
-rw-r--r-- 1 cassandra cassandra 56 Dec 11 06:10 mc-79-big-Summary.db
-rw-r--r-- 1 cassandra cassandra 92 Dec 11 06:10 mc-79-big-TOC.txt
-rw-r--r-- 1 cassandra cassandra 51 Dec 11 06:10 mc-80-big-CompressionInfo.db
-rw-r--r-- 1 cassandra cassandra 2327 Dec 11 06:10 mc-80-big-Data.db
-rw-r--r-- 1 cassandra cassandra 9 Dec 11 06:10 mc-80-big-Digest.crc32
-rw-r--r-- 1 cassandra cassandra 72 Dec 11 06:10 mc-80-big-Filter.db
-rw-r--r-- 1 cassandra cassandra 430 Dec 11 06:10 mc-80-big-Index.db
-rw-r--r-- 1 cassandra cassandra 4821 Dec 11 06:10 mc-80-big-Statistics.db
-rw-r--r-- 1 cassandra cassandra 56 Dec 11 06:10 mc-80-big-Summary.db
-rw-r--r-- 1
[jira] [Commented] (CASSANDRA-14880) drop table and materialized view frequently get error over time
[
https://issues.apache.org/jira/browse/CASSANDRA-14880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716105#comment-16716105
]
jhon commented on CASSANDRA-14880:
--
in fact,before delete materialized views ,I will add some data to table and
sleep some time to wait it delete success.I find some delete operation will
take 10 seconds if delete materialized views frequently, and I set cassandra
operation timeout from 10s to 30s, it seems no problems.
> drop table and materialized view frequently get error over time
> ---
>
> Key: CASSANDRA-14880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14880
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
>Reporter: jhon
>Assignee: jhon
>Priority: Major
>
> when i create table and materialized view,then i drop it, if i drop it
> frequently it will got error: "no response received from cassandra within
> timeout period",for example：
> for i=0; i<100; i++ {
> create table;
> create materialized view;
> drop materialized view;
> drop table;
> how can i solve it ?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14928) MigrationManager attempts to pull schema from different major version nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715633#comment-16715633 ] Ariel Weisberg commented on CASSANDRA-14928: ||Branch|CircleCI| |[2.1|https://github.com/apache/cassandra/compare/cassandra-2.1...aweisberg:14928-2.1?expand=1]|[CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/14928-2%2E1]| |[2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...aweisberg:14928-2.2?expand=1]|[CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/14928-2%2E2]| |[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...aweisberg:14928-3.0?expand=1]|[CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/14928-3%2E0]| |[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...aweisberg:14928-3.11?expand=1]|[CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/14928-3%2E11]| |[trunk|https://github.com/apache/cassandra/compare/cassandra-2.2...aweisberg:14928-trunk?expand=1]|[CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/14928-trunk]| > MigrationManager attempts to pull schema from different major version nodes > --- > > Key: CASSANDRA-14928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14928 > Project: Cassandra > Issue Type: Bug > Components: 4.0, Distributed Metadata >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Blocker > Fix For: 4.0, 3.0.x, 3.11.x > > > MigrationManager will do the version check against nodes it hasn't connected > to yet so it doesn't know their messaging service version. We should also > check the version in gossip as an additional layer of protection. > This causes many of the upgrade tests to fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14923) Java8 forEach cannot be used in UDF
[
https://issues.apache.org/jira/browse/CASSANDRA-14923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
C. Scott Andreas resolved CASSANDRA-14923.
--
Resolution: Duplicate
> Java8 forEach cannot be used in UDF
> ---
>
> Key: CASSANDRA-14923
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14923
> Project: Cassandra
> Issue Type: Bug
> Environment: FreeBSD 11.2
> Cassandra 3.11.3
> OpenJDK 8.181.13
>Reporter: Lapo Luchini
>Priority: Major
>
> I get the following error:
> {noformat}
> cqlsh:test> CREATE OR REPLACE FUNCTION sumMap (state map, val
> map)
> ... RETURNS NULL ON NULL INPUT
> ... RETURNS map
> ... LANGUAGE java AS
> ... $$
> ... val.forEach((k, v) -> {
> ... Integer cur = state.get(k);
> ... state.put(k, (cur == null) ? v : cur + v);
> ... });
> ... return state;
> ... $$;
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Java
> source compilation failed:
> Line 2: The type java.util.function.BiConsumer cannot be resolved. It is
> indirectly referenced from required .class files
> Line 2: The method forEach(BiConsumer) from
> the type Map refers to the missing type BiConsumer
> Line 2: The target type of this expression must be a functional interface
> "
> {noformat}
> on the other hand, this compiles correctly:
> {noformat}
> CREATE OR REPLACE FUNCTION sumMap (state map, val map)
> RETURNS NULL ON NULL INPUT
> RETURNS map
> LANGUAGE java AS
> $$
> for (Map.Entry e : val.entrySet()) {
> String k = e.getKey();
> Integer v = e.getValue();
> Integer cur = state.get(k);
> state.put(k, (cur == null) ? v : cur + v);
> };
> return state;
> $$;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14923) Java8 forEach cannot be used in UDF
[
https://issues.apache.org/jira/browse/CASSANDRA-14923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715623#comment-16715623
]
C. Scott Andreas commented on CASSANDRA-14923:
--
Thanks for filing, [~l...@lapo.it]! This is tracked under
[CASSANDRA-11052|https://issues.apache.org/jira/browse/CASSANDRA-11052].
> Java8 forEach cannot be used in UDF
> ---
>
> Key: CASSANDRA-14923
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14923
> Project: Cassandra
> Issue Type: Bug
> Environment: FreeBSD 11.2
> Cassandra 3.11.3
> OpenJDK 8.181.13
>Reporter: Lapo Luchini
>Priority: Major
>
> I get the following error:
> {noformat}
> cqlsh:test> CREATE OR REPLACE FUNCTION sumMap (state map, val
> map)
> ... RETURNS NULL ON NULL INPUT
> ... RETURNS map
> ... LANGUAGE java AS
> ... $$
> ... val.forEach((k, v) -> {
> ... Integer cur = state.get(k);
> ... state.put(k, (cur == null) ? v : cur + v);
> ... });
> ... return state;
> ... $$;
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Java
> source compilation failed:
> Line 2: The type java.util.function.BiConsumer cannot be resolved. It is
> indirectly referenced from required .class files
> Line 2: The method forEach(BiConsumer) from
> the type Map refers to the missing type BiConsumer
> Line 2: The target type of this expression must be a functional interface
> "
> {noformat}
> on the other hand, this compiles correctly:
> {noformat}
> CREATE OR REPLACE FUNCTION sumMap (state map, val map)
> RETURNS NULL ON NULL INPUT
> RETURNS map
> LANGUAGE java AS
> $$
> for (Map.Entry e : val.entrySet()) {
> String k = e.getKey();
> Integer v = e.getValue();
> Integer cur = state.get(k);
> state.put(k, (cur == null) ? v : cur + v);
> };
> return state;
> $$;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14928) MigrationManager attempts to pull schema from different major version nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14928: --- Description: MigrationManager will do the version check against nodes it hasn't connected to yet so it doesn't know their messaging service version. We should also check the version in gossip as an additional layer of protection. This causes many of the upgrade tests to fail. was: MigrationManager will do the version check against nodes it hasn't connected to yet so it doesn't know their messaging service version. We should also check the version in gossip as an additional layer of protection. This causes many of the upgrade tests to fail as the exceptions from the failed schema pull are detected. > MigrationManager attempts to pull schema from different major version nodes > --- > > Key: CASSANDRA-14928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14928 > Project: Cassandra > Issue Type: Bug > Components: 4.0, Distributed Metadata >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Blocker > Fix For: 4.0, 3.0.x, 3.11.x > > > MigrationManager will do the version check against nodes it hasn't connected > to yet so it doesn't know their messaging service version. We should also > check the version in gossip as an additional layer of protection. > This causes many of the upgrade tests to fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14928) MigrationManager attempts to pull schema from different major version nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14928: --- Description: MigrationManager will do the version check against nodes it hasn't connected to yet so it doesn't know their messaging service version. We should also check the version in gossip as an additional layer of protection. This causes many of the upgrade tests to fail as the exceptions from the failed schema pull are detected. was:MigrationManager will do the version check against nodes it hasn't connected to yet so it doesn't know their messaging service version. We should also check the version in gossip as an additional layer of protection. > MigrationManager attempts to pull schema from different major version nodes > --- > > Key: CASSANDRA-14928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14928 > Project: Cassandra > Issue Type: Bug > Components: 4.0, Distributed Metadata >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Blocker > Fix For: 4.0, 3.0.x, 3.11.x > > > MigrationManager will do the version check against nodes it hasn't connected > to yet so it doesn't know their messaging service version. We should also > check the version in gossip as an additional layer of protection. > This causes many of the upgrade tests to fail as the exceptions from the > failed schema pull are detected. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14928) MigrationManager attempts to pull schema from different major version nodes
Ariel Weisberg created CASSANDRA-14928: -- Summary: MigrationManager attempts to pull schema from different major version nodes Key: CASSANDRA-14928 URL: https://issues.apache.org/jira/browse/CASSANDRA-14928 Project: Cassandra Issue Type: Bug Components: 4.0, Distributed Metadata Reporter: Ariel Weisberg Assignee: Ariel Weisberg Fix For: 4.0, 3.0.x, 3.11.x MigrationManager will do the version check against nodes it hasn't connected to yet so it doesn't know their messaging service version. We should also check the version in gossip as an additional layer of protection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14927) During data migration from 7 node to 21 node cluster using sstableloader, new data is being populated on the new tables & data is being duplicated on user type table
[
https://issues.apache.org/jira/browse/CASSANDRA-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KALYAN CHAKRAVARTHY KANCHARLA updated CASSANDRA-14927:
--
Priority: Blocker (was: Major)
> During data migration from 7 node to 21 node cluster using sstableloader, new
> data is being populated on the new tables & data is being duplicated on user
> type tables
> ---
>
> Key: CASSANDRA-14927
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14927
> Project: Cassandra
> Issue Type: Test
>Reporter: KALYAN CHAKRAVARTHY KANCHARLA
>Priority: Blocker
> Labels: test
> Fix For: 2.1.13
>
>
> I'm trying to migrate data from 7 node (single DC) cluster to a 21 node (3
> DC) cluster using sstableloader.
> We have same versions on both old and new clusters.
> *cqlsh 5.0.1*
> *Cassandra 2.1.13*
> *CQL spec 3.2.1*
> Old and New clusters are in different networks. So we opened the following
> ports between them.
> 7000- storage port
> 7001- ssl storage port
> 7199- JMX port
> 9042- client port
> 9160- Thrift client port
> We use vnodes in the clusters.
> We made sure cassandra.yaml file on the new cluster is set correct by
> changing following options,
>
> {{cluster_name: 'MyCassandraCluster' }}
> {{num_tokens: 256 }}
> {{seed_provider: - }}
> {{class_name: org.apache.cassandra.locator.SimpleSeedProvider }}
> {{parameters: - }}
> {{seeds: "10.168.66.41,10.176.170.59" }}
> {{listen_address: localhost}}
> {{endpoint_snitch: GossipingPropertyFileSnitch}}
> And also changes in cassaandra-rackdc-properties for each DC by specifying
> respective DC and rack.
> while creating keyspaces, changed Replication to NetworkTopologyStratagy.
>
> cluster looks healthy, all the node are UP and NORMAL.
>
> {color:#FF}*I was able to get the data from old cluster to new cluster.
> But, along with the data from old cluster, I see some new rows being
> populated in the tables on new cluster and data is being duplicated in the
> tables with user type*. {color}
> {color:#33}We have used the following steps to migrate data:{color}
> # Took snapshorts for all the keyspaces that we want to migrate. (9
> keyspaces). Used the _nodetool snapshot_ command on source nodes to take
> snapshot of required keyspace/table by specifying _hostname, jmx port_ and
> _keyspace_
> __
> _/a/cassandra/bin/nodetool -u $(sudo su - company -c "cat
> /a/cassandra/jmxremote.password" | awk '\{print $1}') -pw $(sudo su - company
> -c "cat /a/cassandra/jmxremote.password" | awk '\{print $2}')_ *_-h
> localhost -p 7199 snapshot keyspace_name_*
> # After taking snapshots, move these snapshot directory from source nodes to
> target node.
>
> → Create a tar file on source node for the snapshot directory that we want to
> move on to target node.
> tar -cvf file.tar snapshot_name
> → Move this file.tar from source node to local machine.
> scp -S gwsh root@192.168.64.99:/a/cassandra/data/file.tar .
> → Now move this file.tar from local machine to a new directory(example: test)
> in the target node.
> scp -S gwsh file.tar root@192.168.58.41:/a/cassandra/data/test/.
> # Now untar this file.tar in test directory in target node.
> # The path of the sstables must be same in both source and target.
> # To bulk load these files using _sstableloader,run sstableloader on source
> node, indicate one or more nodes in the destination Cluster with -d flag,
> which can accept comma-separated list of IP addresses or hostnames, and
> specify the path to sstables in the source node._ __
> _/a/Cassandra/bin/_ *_./sstableloader -d host_IP path_to_sstables_*
> *_Example:_*
> [/a/cassandra/bin#|mailto:root@sqa-cassandra03.sqaextranet:/a/cassandra/bin]
> sstableloader -d 192.168.58.41 -u popps -pw *** -tf
> org.apache.cassandra.thrift.SSLTransportFactory -ts
> /a/cassandra/ssl/truststore.jks -tspw test123 -ks
> /a/cassandra/ssl/keystore.jks -kspw test123 -f
> /a/cassandra/conf/cassandra.yaml
> /a/cassandra/data/app_properties/_admins-58524140431511e8bbb6357f562e11ca_/
> Summary statistics:
> Connections per host: : 1
> Total files transferred: : 9
> Total bytes transferred: : 1787893
> Total duration (ms): : 2936
> Average transfer rate (MB/s): : 0
> Peak transfer rate (MB/s): : 0
>
> Performed these steps on all the tables. And checked the row count in old and
> new tables using CQLSH
> cqlsh> SELECT count(*) FROM keyspace.table;
> example for a single table:
> count on new table: 341
> count on old table: 303
>
> And we are also able to identify the difference in tables by using 'sdiff'
> command. Followed the following steps:
> * created
[jira] [Commented] (CASSANDRA-14910) Don't skip entire sstables when reading backwards with mixed clustering column order
[
https://issues.apache.org/jira/browse/CASSANDRA-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715312#comment-16715312
]
Aleksey Yeschenko commented on CASSANDRA-14910:
---
Committed to 2.2 as
[afa4563864889c78569e29466047b411cd866b38|https://github.com/apache/cassandra/commit/afa4563864889c78569e29466047b411cd866b38]
and merged (the unit tests) upwards. Thanks!
> Don't skip entire sstables when reading backwards with mixed clustering
> column order
>
>
> Key: CASSANDRA-14910
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14910
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 2.2.14
>
>
> In 2.x, if a table has clustering columns in {{DESC}} order, any SSTable that
> doesn’t have any static rows in it will be skipped altogether when iterated
> in reverse.
> This occurs due to the logic in {{ColumnSlice.compare()}} errorneusly sorting
> any empty {{ByteBuffer}} after non-empty values due to the way
> {{ReversedType}} operates. In case that empty {{ByteBuffer}} is coming from a
> static {{Composite}}, however, the logic breaks down. Static {{Composite}}
> components must *always* sort before any non-empty value, no matter the
> table’s comparator.
> 2.0, 2.1, and 2.2 are all affected. 3.0 isn’t, but only because we skip slice
> intersection logic entirely if static rows are present in the filter.
> Introduced by CASSANDRA-8502.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14910) Don't skip entire sstables when reading backwards with mixed clustering column order
[
https://issues.apache.org/jira/browse/CASSANDRA-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14910:
--
Resolution: Fixed
Fix Version/s: (was: 2.2.x)
2.2.14
Reproduced In: 2.2.13, 2.1.20 (was: 2.1.20, 2.2.13)
Status: Resolved (was: Ready to Commit)
> Don't skip entire sstables when reading backwards with mixed clustering
> column order
>
>
> Key: CASSANDRA-14910
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14910
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 2.2.14
>
>
> In 2.x, if a table has clustering columns in {{DESC}} order, any SSTable that
> doesn’t have any static rows in it will be skipped altogether when iterated
> in reverse.
> This occurs due to the logic in {{ColumnSlice.compare()}} errorneusly sorting
> any empty {{ByteBuffer}} after non-empty values due to the way
> {{ReversedType}} operates. In case that empty {{ByteBuffer}} is coming from a
> static {{Composite}}, however, the logic breaks down. Static {{Composite}}
> components must *always* sort before any non-empty value, no matter the
> table’s comparator.
> 2.0, 2.1, and 2.2 are all affected. 3.0 isn’t, but only because we skip slice
> intersection logic entirely if static rows are present in the filter.
> Introduced by CASSANDRA-8502.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[03/10] cassandra git commit: Don't skip entire sstables when reading backwards with mixed clustering column order
Don't skip entire sstables when reading backwards with mixed clustering column
order
patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-14910
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/afa45638
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/afa45638
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/afa45638
Branch: refs/heads/cassandra-3.11
Commit: afa4563864889c78569e29466047b411cd866b38
Parents: cf6f792
Author: Aleksey Yeshchenko
Authored: Thu Nov 22 14:59:22 2018 +
Committer: Aleksey Yeshchenko
Committed: Tue Nov 27 21:03:56 2018 +
--
CHANGES.txt | 2 +
.../apache/cassandra/db/filter/ColumnSlice.java | 5 +-
.../operations/SelectOrderByTest.java | 78
3 files changed, 84 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index bca036d..b989f3c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
2.2.14
+ * Don't skip entire sstables when reading backwards with mixed clustering
column order
+ (CASSANDRA-14910)
* Cannot perform slice reads in reverse direction against tables with
clustering columns
in mixed order (CASSANDRA-14899)
* Fix incorrect cqlsh results when selecting same columns multiple times
(CASSANDRA-13262)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
index 1cc348c..316226d 100644
--- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
@@ -68,7 +68,10 @@ public class ColumnSlice
Composite sStart = reversed ? finish : start;
Composite sEnd = reversed ? start : finish;
-if (compare(sStart, maxCellNames, comparator, true) > 0 ||
compare(sEnd, minCellNames, comparator, false) < 0)
+// don't compare static slice bounds with min/max cell names to
determine intersection - that can yield unexpected
+// results, in particular with ReverseType comparators; see
CASSANDRA-14910 for more context.
+if ((!sStart.isStatic() && compare(sStart, maxCellNames, comparator,
true) > 0)
+ || (!sEnd.isStatic() && compare(sEnd, minCellNames, comparator,
false) < 0))
return false;
// We could safely return true here, but there's a minor optimization:
if the first component is restricted
http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
index 32d800a..e21074b 100644
---
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
+++
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
@@ -732,6 +732,84 @@ public class SelectOrderByTest extends CQLTester
}
}
+@Test
+public void
testSelectWithReversedTypeInReverseOrderWithStaticColumnsWithoutStaticRow()
throws Throwable
+{
+createTable("CREATE TABLE %s (a int, b int, c int, d int static,
PRIMARY KEY (a, b)) WITH CLUSTERING ORDER BY (b DESC);");
+
+execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 1);");
+execute("INSERT INTO %s (a, b, c) VALUES (1, 2, 2);");
+execute("INSERT INTO %s (a, b, c) VALUES (1, 3, 3);");
+
+// read in comparator order
+assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"),
+ row(3, 3),
+ row(2, 2),
+ row(1, 1));
+
+// read in reverse comparator order
+assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b ASC;"),
+ row(1, 1),
+ row(2, 2),
+ row(3, 3));
+
+/*
+ * Flush the sstable. We *should* see the same results when reading in
both directions, but prior to CASSANDRA-14910
+ * fix this would now have returned an empty result set when reading
in reverse comparator order.
+ */
+flush();
+
+// read in comparator order
+assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"),
+ row(3, 3),
+
[10/10] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b871decb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b871decb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b871decb
Branch: refs/heads/trunk
Commit: b871decba274be126fc94ab3ab48f75a95be9d94
Parents: 33eada0 2edd6fd
Author: Aleksey Yeshchenko
Authored: Mon Dec 10 18:23:39 2018 +
Committer: Aleksey Yeshchenko
Committed: Mon Dec 10 18:23:39 2018 +
--
.../operations/SelectOrderByTest.java | 78
1 file changed, 78 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b871decb/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
--
diff --cc
test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
index fd44c50,06aa2fd..8a3ae03
---
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
+++
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
@@@ -639,132 -702,84 +639,210 @@@ public class SelectOrderByTest extends
}
}
+/**
+ * Test that ORDER BY columns allow skipping equality-restricted
clustering columns, see CASSANDRA-10271.
+ */
+@Test
+public void
testAllowSkippingEqualityAndSingleValueInRestrictedClusteringColumns() throws
Throwable
+{
+createTable("CREATE TABLE %s (a int, b int, c int, d int, PRIMARY KEY
(a, b, c))");
+execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 0, 0,
0);
+execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 0, 1,
1);
+execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 0, 2,
2);
+execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 1, 0,
3);
+execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 1, 1,
4);
+execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 1, 2,
5);
+
+assertInvalidMessage("Order by is currently only supported on the
clustered columns of the PRIMARY KEY, got d",
+ "SELECT * FROM %s WHERE a=? ORDER BY d DESC", 0);
+
+assertInvalidMessage("Order by is currently only supported on the
clustered columns of the PRIMARY KEY, got d",
+ "SELECT * FROM %s WHERE a=? ORDER BY b ASC, c
ASC, d ASC", 0);
+
+String errorMsg = "Order by currently only supports the ordering of
columns following their declared order in the PRIMARY KEY";
+
+assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? ORDER BY c",
0, 0),
+ row(0, 0, 0, 0),
+ row(0, 0, 1, 1),
+ row(0, 0, 2, 2)
+);
+
+assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? ORDER BY c
ASC", 0, 0),
+ row(0, 0, 0, 0),
+ row(0, 0, 1, 1),
+ row(0, 0, 2, 2)
+);
+
+assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? ORDER BY c
DESC", 0, 0),
+ row(0, 0, 2, 2),
+ row(0, 0, 1, 1),
+ row(0, 0, 0, 0)
+);
+
+assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? AND c>=? ORDER
BY c ASC", 0, 0, 1),
+ row(0, 0, 1, 1),
+ row(0, 0, 2, 2));
+
+assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? AND c>=? ORDER
BY c DESC", 0, 0, 1),
+ row(0, 0, 2, 2),
+ row(0, 0, 1, 1));
+
+assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? AND c IN (?,
?) ORDER BY c ASC", 0, 0, 1, 2),
+ row(0, 0, 1, 1),
+ row(0, 0, 2, 2));
+
+assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? AND c IN (?,
?) ORDER BY c DESC", 0, 0, 1, 2),
+ row(0, 0, 2, 2),
+ row(0, 0, 1, 1));
+
+assertInvalidMessage(errorMsg, "SELECT * FROM %s WHERE a=? AND b (?, ?) ORDER BY c", 0, 0, 0);
+assertInvalidMessage(errorMsg, "SELECT * FROM %s WHERE a=? AND (b, c)
>= (?, ?) ORDER BY c", 0, 0, 0);
+assertInvalidMessage(errorMsg, "SELECT * FROM %s WHERE a=? AND (b, c)
< (?, ?) ORDER BY c", 0, 0, 0);
+assertInvalidMessage(errorMsg, "SELECT * FROM %s WHERE a=? AND (b, c)
<= (?, ?) ORDER BY c", 0, 0, 0);
+assertRows(execute("SELECT * FROM %s WHERE a=? AND (b, c) = (?, ?)
ORDER BY c ASC", 0, 0, 0),
+ row(0, 0, 0, 0));
+assertRows(execute("SELECT * FROM %s WHERE a=? AND (b, c) = (?, ?)
ORDER BY c DESC", 0, 0, 0),
+ row(0, 0, 0, 0));
+
+
[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4f3a6feb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4f3a6feb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4f3a6feb Branch: refs/heads/cassandra-3.0 Commit: 4f3a6feb22f844a99e1c20c20ae17b2427f81307 Parents: 84ffcb8 afa4563 Author: Aleksey Yeshchenko Authored: Mon Dec 10 18:14:14 2018 + Committer: Aleksey Yeshchenko Committed: Mon Dec 10 18:14:14 2018 + -- .../operations/SelectOrderByTest.java | 78 1 file changed, 78 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f3a6feb/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[04/10] cassandra git commit: Don't skip entire sstables when reading backwards with mixed clustering column order
Don't skip entire sstables when reading backwards with mixed clustering column
order
patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-14910
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/afa45638
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/afa45638
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/afa45638
Branch: refs/heads/trunk
Commit: afa4563864889c78569e29466047b411cd866b38
Parents: cf6f792
Author: Aleksey Yeshchenko
Authored: Thu Nov 22 14:59:22 2018 +
Committer: Aleksey Yeshchenko
Committed: Tue Nov 27 21:03:56 2018 +
--
CHANGES.txt | 2 +
.../apache/cassandra/db/filter/ColumnSlice.java | 5 +-
.../operations/SelectOrderByTest.java | 78
3 files changed, 84 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index bca036d..b989f3c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
2.2.14
+ * Don't skip entire sstables when reading backwards with mixed clustering
column order
+ (CASSANDRA-14910)
* Cannot perform slice reads in reverse direction against tables with
clustering columns
in mixed order (CASSANDRA-14899)
* Fix incorrect cqlsh results when selecting same columns multiple times
(CASSANDRA-13262)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
index 1cc348c..316226d 100644
--- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
@@ -68,7 +68,10 @@ public class ColumnSlice
Composite sStart = reversed ? finish : start;
Composite sEnd = reversed ? start : finish;
-if (compare(sStart, maxCellNames, comparator, true) > 0 ||
compare(sEnd, minCellNames, comparator, false) < 0)
+// don't compare static slice bounds with min/max cell names to
determine intersection - that can yield unexpected
+// results, in particular with ReverseType comparators; see
CASSANDRA-14910 for more context.
+if ((!sStart.isStatic() && compare(sStart, maxCellNames, comparator,
true) > 0)
+ || (!sEnd.isStatic() && compare(sEnd, minCellNames, comparator,
false) < 0))
return false;
// We could safely return true here, but there's a minor optimization:
if the first component is restricted
http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
index 32d800a..e21074b 100644
---
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
+++
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
@@ -732,6 +732,84 @@ public class SelectOrderByTest extends CQLTester
}
}
+@Test
+public void
testSelectWithReversedTypeInReverseOrderWithStaticColumnsWithoutStaticRow()
throws Throwable
+{
+createTable("CREATE TABLE %s (a int, b int, c int, d int static,
PRIMARY KEY (a, b)) WITH CLUSTERING ORDER BY (b DESC);");
+
+execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 1);");
+execute("INSERT INTO %s (a, b, c) VALUES (1, 2, 2);");
+execute("INSERT INTO %s (a, b, c) VALUES (1, 3, 3);");
+
+// read in comparator order
+assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"),
+ row(3, 3),
+ row(2, 2),
+ row(1, 1));
+
+// read in reverse comparator order
+assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b ASC;"),
+ row(1, 1),
+ row(2, 2),
+ row(3, 3));
+
+/*
+ * Flush the sstable. We *should* see the same results when reading in
both directions, but prior to CASSANDRA-14910
+ * fix this would now have returned an empty result set when reading
in reverse comparator order.
+ */
+flush();
+
+// read in comparator order
+assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"),
+ row(3, 3),
+
[02/10] cassandra git commit: Don't skip entire sstables when reading backwards with mixed clustering column order
Don't skip entire sstables when reading backwards with mixed clustering column
order
patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-14910
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/afa45638
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/afa45638
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/afa45638
Branch: refs/heads/cassandra-3.0
Commit: afa4563864889c78569e29466047b411cd866b38
Parents: cf6f792
Author: Aleksey Yeshchenko
Authored: Thu Nov 22 14:59:22 2018 +
Committer: Aleksey Yeshchenko
Committed: Tue Nov 27 21:03:56 2018 +
--
CHANGES.txt | 2 +
.../apache/cassandra/db/filter/ColumnSlice.java | 5 +-
.../operations/SelectOrderByTest.java | 78
3 files changed, 84 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index bca036d..b989f3c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
2.2.14
+ * Don't skip entire sstables when reading backwards with mixed clustering
column order
+ (CASSANDRA-14910)
* Cannot perform slice reads in reverse direction against tables with
clustering columns
in mixed order (CASSANDRA-14899)
* Fix incorrect cqlsh results when selecting same columns multiple times
(CASSANDRA-13262)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
index 1cc348c..316226d 100644
--- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
@@ -68,7 +68,10 @@ public class ColumnSlice
Composite sStart = reversed ? finish : start;
Composite sEnd = reversed ? start : finish;
-if (compare(sStart, maxCellNames, comparator, true) > 0 ||
compare(sEnd, minCellNames, comparator, false) < 0)
+// don't compare static slice bounds with min/max cell names to
determine intersection - that can yield unexpected
+// results, in particular with ReverseType comparators; see
CASSANDRA-14910 for more context.
+if ((!sStart.isStatic() && compare(sStart, maxCellNames, comparator,
true) > 0)
+ || (!sEnd.isStatic() && compare(sEnd, minCellNames, comparator,
false) < 0))
return false;
// We could safely return true here, but there's a minor optimization:
if the first component is restricted
http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
index 32d800a..e21074b 100644
---
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
+++
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
@@ -732,6 +732,84 @@ public class SelectOrderByTest extends CQLTester
}
}
+@Test
+public void
testSelectWithReversedTypeInReverseOrderWithStaticColumnsWithoutStaticRow()
throws Throwable
+{
+createTable("CREATE TABLE %s (a int, b int, c int, d int static,
PRIMARY KEY (a, b)) WITH CLUSTERING ORDER BY (b DESC);");
+
+execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 1);");
+execute("INSERT INTO %s (a, b, c) VALUES (1, 2, 2);");
+execute("INSERT INTO %s (a, b, c) VALUES (1, 3, 3);");
+
+// read in comparator order
+assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"),
+ row(3, 3),
+ row(2, 2),
+ row(1, 1));
+
+// read in reverse comparator order
+assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b ASC;"),
+ row(1, 1),
+ row(2, 2),
+ row(3, 3));
+
+/*
+ * Flush the sstable. We *should* see the same results when reading in
both directions, but prior to CASSANDRA-14910
+ * fix this would now have returned an empty result set when reading
in reverse comparator order.
+ */
+flush();
+
+// read in comparator order
+assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"),
+ row(3, 3),
+
[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2edd6fd2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2edd6fd2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2edd6fd2 Branch: refs/heads/trunk Commit: 2edd6fd2e02b9e2769e4ea751c1030428f70ce47 Parents: 732c43b 4f3a6fe Author: Aleksey Yeshchenko Authored: Mon Dec 10 18:19:31 2018 + Committer: Aleksey Yeshchenko Committed: Mon Dec 10 18:19:31 2018 + -- .../operations/SelectOrderByTest.java | 78 1 file changed, 78 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2edd6fd2/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2edd6fd2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2edd6fd2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2edd6fd2 Branch: refs/heads/cassandra-3.11 Commit: 2edd6fd2e02b9e2769e4ea751c1030428f70ce47 Parents: 732c43b 4f3a6fe Author: Aleksey Yeshchenko Authored: Mon Dec 10 18:19:31 2018 + Committer: Aleksey Yeshchenko Committed: Mon Dec 10 18:19:31 2018 + -- .../operations/SelectOrderByTest.java | 78 1 file changed, 78 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2edd6fd2/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4f3a6feb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4f3a6feb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4f3a6feb Branch: refs/heads/trunk Commit: 4f3a6feb22f844a99e1c20c20ae17b2427f81307 Parents: 84ffcb8 afa4563 Author: Aleksey Yeshchenko Authored: Mon Dec 10 18:14:14 2018 + Committer: Aleksey Yeshchenko Committed: Mon Dec 10 18:14:14 2018 + -- .../operations/SelectOrderByTest.java | 78 1 file changed, 78 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f3a6feb/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[01/10] cassandra git commit: Don't skip entire sstables when reading backwards with mixed clustering column order
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 cf6f7920f -> afa456386
refs/heads/cassandra-3.0 84ffcb82a -> 4f3a6feb2
refs/heads/cassandra-3.11 732c43b0f -> 2edd6fd2e
refs/heads/trunk 33eada06a -> b871decba
Don't skip entire sstables when reading backwards with mixed clustering column
order
patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-14910
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/afa45638
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/afa45638
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/afa45638
Branch: refs/heads/cassandra-2.2
Commit: afa4563864889c78569e29466047b411cd866b38
Parents: cf6f792
Author: Aleksey Yeshchenko
Authored: Thu Nov 22 14:59:22 2018 +
Committer: Aleksey Yeshchenko
Committed: Tue Nov 27 21:03:56 2018 +
--
CHANGES.txt | 2 +
.../apache/cassandra/db/filter/ColumnSlice.java | 5 +-
.../operations/SelectOrderByTest.java | 78
3 files changed, 84 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index bca036d..b989f3c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
2.2.14
+ * Don't skip entire sstables when reading backwards with mixed clustering
column order
+ (CASSANDRA-14910)
* Cannot perform slice reads in reverse direction against tables with
clustering columns
in mixed order (CASSANDRA-14899)
* Fix incorrect cqlsh results when selecting same columns multiple times
(CASSANDRA-13262)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
index 1cc348c..316226d 100644
--- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
@@ -68,7 +68,10 @@ public class ColumnSlice
Composite sStart = reversed ? finish : start;
Composite sEnd = reversed ? start : finish;
-if (compare(sStart, maxCellNames, comparator, true) > 0 ||
compare(sEnd, minCellNames, comparator, false) < 0)
+// don't compare static slice bounds with min/max cell names to
determine intersection - that can yield unexpected
+// results, in particular with ReverseType comparators; see
CASSANDRA-14910 for more context.
+if ((!sStart.isStatic() && compare(sStart, maxCellNames, comparator,
true) > 0)
+ || (!sEnd.isStatic() && compare(sEnd, minCellNames, comparator,
false) < 0))
return false;
// We could safely return true here, but there's a minor optimization:
if the first component is restricted
http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
index 32d800a..e21074b 100644
---
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
+++
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java
@@ -732,6 +732,84 @@ public class SelectOrderByTest extends CQLTester
}
}
+@Test
+public void
testSelectWithReversedTypeInReverseOrderWithStaticColumnsWithoutStaticRow()
throws Throwable
+{
+createTable("CREATE TABLE %s (a int, b int, c int, d int static,
PRIMARY KEY (a, b)) WITH CLUSTERING ORDER BY (b DESC);");
+
+execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 1);");
+execute("INSERT INTO %s (a, b, c) VALUES (1, 2, 2);");
+execute("INSERT INTO %s (a, b, c) VALUES (1, 3, 3);");
+
+// read in comparator order
+assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"),
+ row(3, 3),
+ row(2, 2),
+ row(1, 1));
+
+// read in reverse comparator order
+assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b ASC;"),
+ row(1, 1),
+ row(2, 2),
+ row(3, 3));
+
+/*
+ * Flush the sstable. We *should* see the same results when reading in
both directions, but prior to CASSANDRA-14910
+ * fix this would now have returned an empty result
[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4f3a6feb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4f3a6feb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4f3a6feb Branch: refs/heads/cassandra-3.11 Commit: 4f3a6feb22f844a99e1c20c20ae17b2427f81307 Parents: 84ffcb8 afa4563 Author: Aleksey Yeshchenko Authored: Mon Dec 10 18:14:14 2018 + Committer: Aleksey Yeshchenko Committed: Mon Dec 10 18:14:14 2018 + -- .../operations/SelectOrderByTest.java | 78 1 file changed, 78 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f3a6feb/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14927) During data migration from 7 node to 21 node cluster using sstableloader, new data is being populated on the new tables & data is being duplicated on user type table
[
https://issues.apache.org/jira/browse/CASSANDRA-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KALYAN CHAKRAVARTHY KANCHARLA updated CASSANDRA-14927:
--
Issue Type: Test (was: Task)
> During data migration from 7 node to 21 node cluster using sstableloader, new
> data is being populated on the new tables & data is being duplicated on user
> type tables
> ---
>
> Key: CASSANDRA-14927
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14927
> Project: Cassandra
> Issue Type: Test
>Reporter: KALYAN CHAKRAVARTHY KANCHARLA
>Priority: Major
> Labels: test
> Fix For: 2.1.13
>
>
> I'm trying to migrate data from 7 node (single DC) cluster to a 21 node (3
> DC) cluster using sstableloader.
> We have same versions on both old and new clusters.
> *cqlsh 5.0.1*
> *Cassandra 2.1.13*
> *CQL spec 3.2.1*
> Old and New clusters are in different networks. So we opened the following
> ports between them.
> 7000- storage port
> 7001- ssl storage port
> 7199- JMX port
> 9042- client port
> 9160- Thrift client port
> We use vnodes in the clusters.
> We made sure cassandra.yaml file on the new cluster is set correct by
> changing following options,
>
> {{cluster_name: 'MyCassandraCluster' }}
> {{num_tokens: 256 }}
> {{seed_provider: - }}
> {{class_name: org.apache.cassandra.locator.SimpleSeedProvider }}
> {{parameters: - }}
> {{seeds: "10.168.66.41,10.176.170.59" }}
> {{listen_address: localhost}}
> {{endpoint_snitch: GossipingPropertyFileSnitch}}
> And also changes in cassaandra-rackdc-properties for each DC by specifying
> respective DC and rack.
> while creating keyspaces, changed Replication to NetworkTopologyStratagy.
>
> cluster looks healthy, all the node are UP and NORMAL.
>
> {color:#FF}*I was able to get the data from old cluster to new cluster.
> But, along with the data from old cluster, I see some new rows being
> populated in the tables on new cluster and data is being duplicated in the
> tables with user type*. {color}
> {color:#33}We have used the following steps to migrate data:{color}
> # Took snapshorts for all the keyspaces that we want to migrate. (9
> keyspaces). Used the _nodetool snapshot_ command on source nodes to take
> snapshot of required keyspace/table by specifying _hostname, jmx port_ and
> _keyspace_
> __
> _/a/cassandra/bin/nodetool -u $(sudo su - company -c "cat
> /a/cassandra/jmxremote.password" | awk '\{print $1}') -pw $(sudo su - company
> -c "cat /a/cassandra/jmxremote.password" | awk '\{print $2}')_ *_-h
> localhost -p 7199 snapshot keyspace_name_*
> # After taking snapshots, move these snapshot directory from source nodes to
> target node.
>
> → Create a tar file on source node for the snapshot directory that we want to
> move on to target node.
> tar -cvf file.tar snapshot_name
> → Move this file.tar from source node to local machine.
> scp -S gwsh root@192.168.64.99:/a/cassandra/data/file.tar .
> → Now move this file.tar from local machine to a new directory(example: test)
> in the target node.
> scp -S gwsh file.tar root@192.168.58.41:/a/cassandra/data/test/.
> # Now untar this file.tar in test directory in target node.
> # The path of the sstables must be same in both source and target.
> # To bulk load these files using _sstableloader,run sstableloader on source
> node, indicate one or more nodes in the destination Cluster with -d flag,
> which can accept comma-separated list of IP addresses or hostnames, and
> specify the path to sstables in the source node._ __
> _/a/Cassandra/bin/_ *_./sstableloader -d host_IP path_to_sstables_*
> *_Example:_*
> [/a/cassandra/bin#|mailto:root@sqa-cassandra03.sqaextranet:/a/cassandra/bin]
> sstableloader -d 192.168.58.41 -u popps -pw *** -tf
> org.apache.cassandra.thrift.SSLTransportFactory -ts
> /a/cassandra/ssl/truststore.jks -tspw test123 -ks
> /a/cassandra/ssl/keystore.jks -kspw test123 -f
> /a/cassandra/conf/cassandra.yaml
> /a/cassandra/data/app_properties/_admins-58524140431511e8bbb6357f562e11ca_/
> Summary statistics:
> Connections per host: : 1
> Total files transferred: : 9
> Total bytes transferred: : 1787893
> Total duration (ms): : 2936
> Average transfer rate (MB/s): : 0
> Peak transfer rate (MB/s): : 0
>
> Performed these steps on all the tables. And checked the row count in old and
> new tables using CQLSH
> cqlsh> SELECT count(*) FROM keyspace.table;
> example for a single table:
> count on new table: 341
> count on old table: 303
>
> And we are also able to identify the difference in tables by using 'sdiff'
> command. Followed the following steps:
> * created
[jira] [Created] (CASSANDRA-14927) During data migration from 7 node to 21 node cluster using sstableloader, new data is being populated on the new tables & data is being duplicated on user type table
KALYAN CHAKRAVARTHY KANCHARLA created CASSANDRA-14927:
-
Summary: During data migration from 7 node to 21 node cluster
using sstableloader, new data is being populated on the new tables & data is
being duplicated on user type tables
Key: CASSANDRA-14927
URL: https://issues.apache.org/jira/browse/CASSANDRA-14927
Project: Cassandra
Issue Type: Task
Reporter: KALYAN CHAKRAVARTHY KANCHARLA
Fix For: 2.1.13
I'm trying to migrate data from 7 node (single DC) cluster to a 21 node (3 DC)
cluster using sstableloader.
We have same versions on both old and new clusters.
*cqlsh 5.0.1*
*Cassandra 2.1.13*
*CQL spec 3.2.1*
Old and New clusters are in different networks. So we opened the following
ports between them.
7000- storage port
7001- ssl storage port
7199- JMX port
9042- client port
9160- Thrift client port
We use vnodes in the clusters.
We made sure cassandra.yaml file on the new cluster is set correct by changing
following options,
{{cluster_name: 'MyCassandraCluster' }}
{{num_tokens: 256 }}
{{seed_provider: - }}
{{class_name: org.apache.cassandra.locator.SimpleSeedProvider }}
{{parameters: - }}
{{seeds: "10.168.66.41,10.176.170.59" }}
{{listen_address: localhost}}
{{endpoint_snitch: GossipingPropertyFileSnitch}}
And also changes in cassaandra-rackdc-properties for each DC by specifying
respective DC and rack.
while creating keyspaces, changed Replication to NetworkTopologyStratagy.
cluster looks healthy, all the node are UP and NORMAL.
{color:#FF}*I was able to get the data from old cluster to new cluster.
But, along with the data from old cluster, I see some new rows being populated
in the tables on new cluster and data is being duplicated in the tables with
user type*. {color}
{color:#33}We have used the following steps to migrate data:{color}
# Took snapshorts for all the keyspaces that we want to migrate. (9
keyspaces). Used the _nodetool snapshot_ command on source nodes to take
snapshot of required keyspace/table by specifying _hostname, jmx port_ and
_keyspace_
__
_/a/cassandra/bin/nodetool -u $(sudo su - company -c "cat
/a/cassandra/jmxremote.password" | awk '\{print $1}') -pw $(sudo su - company
-c "cat /a/cassandra/jmxremote.password" | awk '\{print $2}')_ *_-h localhost
-p 7199 snapshot keyspace_name_*
# After taking snapshots, move these snapshot directory from source nodes to
target node.
→ Create a tar file on source node for the snapshot directory that we want to
move on to target node.
tar -cvf file.tar snapshot_name
→ Move this file.tar from source node to local machine.
scp -S gwsh root@192.168.64.99:/a/cassandra/data/file.tar .
→ Now move this file.tar from local machine to a new directory(example: test)
in the target node.
scp -S gwsh file.tar root@192.168.58.41:/a/cassandra/data/test/.
# Now untar this file.tar in test directory in target node.
# The path of the sstables must be same in both source and target.
# To bulk load these files using _sstableloader,run sstableloader on source
node, indicate one or more nodes in the destination Cluster with -d flag, which
can accept comma-separated list of IP addresses or hostnames, and specify the
path to sstables in the source node._ __
_/a/Cassandra/bin/_ *_./sstableloader -d host_IP path_to_sstables_*
*_Example:_*
[/a/cassandra/bin#|mailto:root@sqa-cassandra03.sqaextranet:/a/cassandra/bin]
sstableloader -d 192.168.58.41 -u popps -pw *** -tf
org.apache.cassandra.thrift.SSLTransportFactory -ts
/a/cassandra/ssl/truststore.jks -tspw test123 -ks /a/cassandra/ssl/keystore.jks
-kspw test123 -f /a/cassandra/conf/cassandra.yaml
/a/cassandra/data/app_properties/_admins-58524140431511e8bbb6357f562e11ca_/
Summary statistics:
Connections per host: : 1
Total files transferred: : 9
Total bytes transferred: : 1787893
Total duration (ms): : 2936
Average transfer rate (MB/s): : 0
Peak transfer rate (MB/s): : 0
Performed these steps on all the tables. And checked the row count in old and
new tables using CQLSH
cqlsh> SELECT count(*) FROM keyspace.table;
example for a single table:
count on new table: 341
count on old table: 303
And we are also able to identify the difference in tables by using 'sdiff'
command. Followed the following steps:
* created .txt/.csv files for tables in old and new clusters.
* compared them using sdiff command
*So I request someone can help me to know the cause behind the population of
new data in the new tables.*
Please let me know if you need more info.
PS: After migrating the data for the first time and saw these issues, we have
TRUNCATED all the tables and DROPPED tables with user 'type' and recreated the
dropped tables. And did the same procedure for migrating data again. Still we
see the same issues.
--
This message
[jira] [Commented] (CASSANDRA-14924) Cassandra nodes becomes unreachable to each other
[
https://issues.apache.org/jira/browse/CASSANDRA-14924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715238#comment-16715238
]
C. Scott Andreas commented on CASSANDRA-14924:
--
Hi [~venksta], thanks for your report. This bug tracker is primarily used by
contributors of the Apache Cassandra project toward development of the database
itself. Can you reach out to the user's list or public IRC channel for support?
A member of the community may be able to help.
Here's a page with information on the best channels for support:
http://cassandra.apache.org/community/
> Cassandra nodes becomes unreachable to each other
> -
>
> Key: CASSANDRA-14924
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14924
> Project: Cassandra
> Issue Type: Bug
>Reporter: ventsislav
>Priority: Critical
>
> I have 3 nodes of elassandra running in docker containers.
> Containers created like:
> {code:java}
> > Host 10.0.0.1 : docker run --name elassandra-node-1 --net=host -e
> > CASSANDRA_SEEDS="10.0.0.1" -e CASSANDRA_CLUSTER_NAME="BD Storage" -e
> > CASSANDRA_DC="DC1" -e CASSANDRA_RACK="r1" -d
> > strapdata/elassandra:latest{code}
> {code:java}
> > Host 10.0.0.2 : docker run --name elassandra-node-2 --net=host -e
> > CASSANDRA_SEEDS="10.0.0.1,10.0.0.2" -e CASSANDRA_CLUSTER_NAME="BD Storage"
> > -e CASSANDRA_DC="DC1" -e CASSANDRA_RACK="r1" -d
> > strapdata/elassandra:latest{code}
> {code:java}
> > Host 10.0.0.3 : docker run --name elassandra-node-3 --net=host -e
> > CASSANDRA_SEEDS="10.0.0.1,10.0.0.2,10.0.0.3" -e CASSANDRA_CLUSTER_NAME="BD
> > Storage" -e CASSANDRA_DC="DC1" -e CASSANDRA_RACK="r1" -d
> > strapdata/elassandra:latest{code}
> Cluster was working fine for a couple of days since created, elastic,
> cassandra all was perfect.
> Currently however all cassandra nodes became unreachable to each other:
> Nodetool status on all nodes is like
> {code:java}
> > Datacenter: DC1
> > ===
> > Status=Up/Down
> > |/ State=Normal/Leaving/Joining/Moving
> > – Address Load Tokens Owns (effective) Host ID Rack
> > DN 10.0.0.3 11.95 GiB 8 100.0% 7652f66e-194e-4886-ac10-0fc21ac8afeb r1
> > DN 10.0.0.2 11.92 GiB 8 100.0% b91fa129-1dd0-4cf8-be96-9c06b23daac6 r1
> > UN 10.0.0.1 11.9 GiB 8 100.0% 5c1afcff-b0aa-4985-a3cc-7f932056c08f r1{code}
> Where the UN is the current host 10.0.0.1
> Same on all other nodes.
> Nodetool describecluster on 10.0.0.1 is like
> {code:java}
> > Cluster Information:
> > Name: BD Storage
> > Snitch: org.apache.cassandra.locator.GossipingPropertyFileSnitch
> > DynamicEndPointSnitch: enabled
> > Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
> > Schema versions:
> > 24fa5e55-3935-3c0e-9808-99ce502fe98d: [10.0.0.1]
> >
> > UNREACHABLE: [10.0.0.2,10.0.0.3]{code}
> When attached to the first node its only repeating these infos:
> {code:java}
> > 2018-12-09 07:47:32,927 WARN [OptionalTasks:1]
> > org.apache.cassandra.auth.CassandraRoleManager.setupDefaultRole(CassandraRoleManager.java:361)
> > CassandraRoleManager skipped default role setup: some nodes were not ready
> > 2018-12-09 07:47:32,927 INFO [OptionalTasks:1]
> org.apache.cassandra.auth.CassandraRoleManager$4.run(CassandraRoleManager.java:400)
> Setup task failed with error, rescheduling
> > 2018-12-09 07:47:32,980 INFO [HANDSHAKE-/10.0.0.2]
> org.apache.cassandra.net.OutboundTcpConnection.lambda$handshakeVersion$1(OutboundTcpConnection.java:561)
> Handshaking version with /10.0.0.2
> > 2018-12-09 07:47:32,980 INFO [HANDSHAKE-/10.0.0.3]
> org.apache.cassandra.net.OutboundTcpConnection.lambda$handshakeVersion$1(OutboundTcpConnection.java:561)
> Handshaking version with /10.0.0.3{code}
> After a while when some node is restarted:
> {code:java}
> > 2018-12-09 07:52:21,972 WARN [MigrationStage:1]
> > org.apache.cassandra.service.MigrationTask.runMayThrow(MigrationTask.java:67)
> > Can't send schema pull request: node /10.0.0.2 is down.{code}
> Tried so far:
> Restarting all containers at the same time
> Restarting all containers one after another
> Restarting cassandra in all containers like : service cassandra restart
> Nodetool disablegossip then enable it
> Nodetool repair : Repair command #1 failed with error Endpoint not alive:
> /10.0.0.2
> Seems that all node schemas are different, but I still dont understand why
> they are marked as down to each other.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14871) Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser
[
https://issues.apache.org/jira/browse/CASSANDRA-14871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714681#comment-16714681
]
Robert Stupp edited comment on CASSANDRA-14871 at 12/10/18 3:59 PM:
I'm fine with using {{Verify}}, but it should include the string that yielded
{{null}}. Like {{Verify.verify(type != null, "Parsing %s yielded null, which is
a bug", str)}} and move that after the assignment of {{type}}. That way we
don't need the {{cache.get(str)}} in the synchronized block, because {{type}}
is then guaranteed to be non-null.
EDIT: So instead of
{code:java}
synchronized (TypeParser.class)
{
if (!cache.containsKey(str))
{
ImmutableMap.Builder> builder =
ImmutableMap.builder();
builder.putAll(cache).put(str, type);
cache = builder.build();
}
AbstractType rtype = cache.get(str);
Verify.verify(rtype != null);
return rtype;
}
{code}
like this:
{code:java}
if (!isEOS(str, i) && str.charAt(i) == '(')
type = getAbstractType(name, new TypeParser(str, i));
else
type = getAbstractType(name);
Verify.verify(type != null, "Parsing %s yielded null, which is a bug",
str);
//
synchronized (TypeParser.class)
{
if (!cache.containsKey(str))
{
ImmutableMap.Builder> builder =
ImmutableMap.builder();
builder.putAll(cache).put(str, type);
cache = builder.build();
}
return type;
}
{code}
was (Author: snazy):
I'm fine with using {{Verify}}, but it should include the string that yielded
{{null}}. Like {{Verify.verify(type != null, "Parsing %s yielded null, which is
a bug", type)}} and move that after the assignment of {{type}}. That way we
don't need the {{cache.get(str)}} in the synchronized block, because {{type}}
is then guaranteed to be non-null.
{code:java}
synchronized (TypeParser.class)
{
if (!cache.containsKey(str))
{
ImmutableMap.Builder> builder =
ImmutableMap.builder();
builder.putAll(cache).put(str, type);
cache = builder.build();
}
AbstractType rtype = cache.get(str);
Verify.verify(rtype != null);
return rtype;
}
{code}
> Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser
> ---
>
> Key: CASSANDRA-14871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14871
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Critical
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
> There are a couple of places in the code base that do not respect that
> j.u.HashMap + related classes are not thread safe and some parts rely on
> internals of the implementation of HM, which can change.
> We have observed failures like {{NullPointerException}} and
> {{ConcurrentModificationException}} as well as wrong behavior.
> Affected areas in the code base:
> * {{SizeTieredCompactionStrategy}}
> * {{DateTieredCompactionStrategy}}
> * {{TimeWindowCompactionStrategy}}
> * {{TokenMetadata.Topology}}
> * {{TypeParser}}
> * streaming / concurrent access to {{LifecycleTransaction}} (handled in
> CASSANDRA-14554)
> While the patches for the compaction strategies + {{TypeParser}} are pretty
> straight forward, the patch for {{TokenMetadata.Topology}} requires it to be
> made immutable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14482) ZSTD Compressor support in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714966#comment-16714966 ] Sushma A Devendrappa commented on CASSANDRA-14482: -- Thanks for all the feedback! [~aweisberg] we have seen similar results as [~jolynch], several usecases show better results than LZ4 and Deflate, ZSTD has good speed for most usecases given same compression ratio. Current code changes does not affect any testing neither touches any other code part of cassandra itself, apart from adding a new compression. So this should be safe changes for any experimentation and also real usage in production. Dictionary based compression is a very good feature of ZSTD, that can be enabled and code changes for that can go in a separate patch in other upcoming releases. Current patch gives a good start for introducing ZSTD as a compression variant and later add more feature. > ZSTD Compressor support in Cassandra > > > Key: CASSANDRA-14482 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14482 > Project: Cassandra > Issue Type: New Feature > Components: Compression, Libraries >Reporter: Sushma A Devendrappa >Assignee: Sushma A Devendrappa >Priority: Major > Labels: performance, pull-request-available > Fix For: 4.x > > Time Spent: 1h 10m > Remaining Estimate: 0h > > ZStandard has a great speed and compression ratio tradeoff. > ZStandard is open source compression from Facebook. > More about ZSTD > [https://github.com/facebook/zstd] > https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/ > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14554) Streaming needs to synchronise access to LifecycleTransaction
[
https://issues.apache.org/jira/browse/CASSANDRA-14554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict resolved CASSANDRA-14554.
--
Resolution: Fixed
Fix Version/s: (was: 3.11.x)
(was: 3.0.x)
3.11.4
3.0.18
Reviewers: Benedict, Robert Stupp (was: Benedict, Dinesh Joshi)
> Streaming needs to synchronise access to LifecycleTransaction
> -
>
> Key: CASSANDRA-14554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14554
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Dinesh Joshi
>Assignee: Stefania
>Priority: Major
> Fix For: 3.0.18, 3.11.4, 4.0
>
>
> When LifecycleTransaction is used in a multi-threaded context, we encounter
> this exception -
> {quote}java.util.ConcurrentModificationException: null
> at
> java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:719)
> at java.util.LinkedHashMap$LinkedKeyIterator.next(LinkedHashMap.java:742)
> at java.lang.Iterable.forEach(Iterable.java:74)
> at
> org.apache.cassandra.db.lifecycle.LogReplicaSet.maybeCreateReplica(LogReplicaSet.java:78)
> at org.apache.cassandra.db.lifecycle.LogFile.makeRecord(LogFile.java:320)
> at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:285)
> at
> org.apache.cassandra.db.lifecycle.LogTransaction.trackNew(LogTransaction.java:136)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.trackNew(LifecycleTransaction.java:529)
> {quote}
> During streaming we create a reference to a {{LifeCycleTransaction}} and
> share it between threads -
> [https://github.com/apache/cassandra/blob/5cc68a87359dd02412bdb70a52dfcd718d44a5ba/src/java/org/apache/cassandra/db/streaming/CassandraStreamReader.java#L156]
> This is used in a multi-threaded context inside {{CassandraIncomingFile}}
> which is an {{IncomingStreamMessage}}. This is being deserialized in parallel.
> {{LifecycleTransaction}} is not meant to be used in a multi-threaded context
> and this leads to streaming failures due to object sharing. On trunk, this
> object is shared across all threads that transfer sstables in parallel for
> the given {{TableId}} in a {{StreamSession}}. There are two options to solve
> this - make {{LifecycleTransaction}} and the associated objects thread safe,
> scope the transaction to a single {{CassandraIncomingFile}}. The consequences
> of the latter option is that if we experience streaming failure we may have
> redundant SSTables on disk. This is ok as compaction should clean this up. A
> third option is we synchronize access in the streaming infrastructure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14926) OpenJDK - Using 1.8.0 "adoptopenjdk" throws error from command line
Aaron Ploetz created CASSANDRA-14926:
Summary: OpenJDK - Using 1.8.0 "adoptopenjdk" throws error from
command line
Key: CASSANDRA-14926
URL: https://issues.apache.org/jira/browse/CASSANDRA-14926
Project: Cassandra
Issue Type: Bug
Components: Configuration
Reporter: Aaron Ploetz
Assignee: Aaron Ploetz
Fix For: 4.x
When testing with a recent build of 4.0, I noticed that Cassandra commands from
my Mac (nodetool, bin/cassandra) was throwing a Bash syntax error:
{{bin/cassandra.in.sh: line 115: [: 1.8.0: integer expression expected}}
The offending line (115 in cassandra.in.sh) is this:
{{if [ "$JVM_PATCH_VERSION" -lt 151 ] ; then}}
Essentially, the patch version of the JDK is being parsed and then verified to
be higher than 1.8.0_151. The value of $JVM_PATCH_VERSION is assumed to be
numeric. Further investigation revealed this to be specific to the OpenJDK
version on my Mac (our test CentOS clusters are fine).
On CentOS, the JDK version reads like this:
{{}}{{$ java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)}}
{{}}But when I check the JDK version on my Mac, I see this:
{{$ java -version
openjdk version "1.8.0-adoptopenjdk"
OpenJDK Runtime Environment (build
1.8.0-adoptopenjdk-jenkins_2018_05_19_02_01-b00)
OpenJDK 64-Bit Server VM (build 25.71-b00, mixed mode)}}
This output causes the $JVM_PATCH_VERSION to resolve to "1.8.0." Obviously not
a valid integer.
I messed around with adding an extra if-check prior to the line in-question:
{{if [[ "$JVM_PATCH_VERSION" =~ ^[0-9]+$ ]] ; then}}
This suppresses the error, but doesn't really validate an adequate patch level
(which is the true goal here). The problem with this OpenJDK version, is that
we'd have to take a different approach in obtaining the patch version. I
wonder how prevalent the "adoptopenjdk" versions are out there to begin with,
and if simply suppressing this error is fine for this edge case? Or is it
worth the extra digging to pull in the patch version in this case?
Note: This is not an issue prior to 4.0, due to the changes made in the
cassandra.in.sh file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[11/11] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/33eada06
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/33eada06
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/33eada06
Branch: refs/heads/trunk
Commit: 33eada06a6dd3529da644377dba180795f522176
Parents: 4dd7faa 732c43b
Author: Benedict Elliott Smith
Authored: Mon Dec 10 15:07:18 2018 +
Committer: Benedict Elliott Smith
Committed: Mon Dec 10 15:07:18 2018 +
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 8 ++--
.../compaction/AbstractCompactionStrategy.java | 5 ++-
.../db/compaction/AbstractStrategyHolder.java | 3 +-
.../db/compaction/CompactionStrategyHolder.java | 5 ++-
.../compaction/CompactionStrategyManager.java | 5 ++-
.../db/compaction/PendingRepairHolder.java | 5 ++-
.../db/lifecycle/LifecycleNewTracker.java | 47
.../db/lifecycle/LifecycleTransaction.java | 7 ++-
.../apache/cassandra/db/lifecycle/LogFile.java | 24 --
.../cassandra/db/lifecycle/LogTransaction.java | 4 +-
.../CassandraEntireSSTableStreamReader.java | 5 ++-
.../db/streaming/CassandraStreamReader.java | 5 ++-
.../db/streaming/CassandraStreamReceiver.java | 47
.../io/sstable/SimpleSSTableMultiWriter.java| 16 +++
.../sstable/format/RangeAwareSSTableWriter.java | 12 ++---
.../io/sstable/format/SSTableWriter.java| 20 -
.../io/sstable/format/big/BigFormat.java| 6 +--
.../io/sstable/format/big/BigTableWriter.java | 6 +--
.../format/big/BigTableZeroCopyWriter.java | 5 ++-
.../locator/AbstractReplicationStrategy.java| 2 +-
.../cassandra/streaming/StreamReceiveTask.java | 2 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 5 ++-
23 files changed, 166 insertions(+), 79 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/33eada06/CHANGES.txt
--
diff --cc CHANGES.txt
index 75b05ea,20da1fa..b8410b8
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -338,7 -2,9 +338,8 @@@
* Make stop-server.bat wait for Cassandra to terminate (CASSANDRA-14829)
* Correct sstable sorting for garbagecollect and levelled compaction
(CASSANDRA-14870)
Merged from 3.0:
+ * Streaming needs to synchronise access to LifecycleTransaction
(CASSANDRA-14554)
* Fix cassandra-stress write hang with default options (CASSANDRA-14616)
- * Differentiate between slices and RTs when decoding legacy bounds
(CASSANDRA-14919)
* Netty epoll IOExceptions caused by unclean client disconnects being logged
at INFO (CASSANDRA-14909)
* Unfiltered.isEmpty conflicts with Row extends AbstractCollection.isEmpty
(CASSANDRA-14588)
* RangeTombstoneList doesn't properly clean up mergeable or superseded rts
in some cases (CASSANDRA-14894)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/33eada06/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index c5149cf,700c1cc..cc6af6f
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -497,15 -507,15 +497,15 @@@ public class ColumnFamilyStore implemen
return directories;
}
- public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, UUID pendingRepair, boolean isTransient, int
sstableLevel, SerializationHeader header, LifecycleTransaction txn)
-public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleNewTracker lifecycleNewTracker)
++public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, UUID pendingRepair, boolean isTransient, int
sstableLevel, SerializationHeader header, LifecycleNewTracker
lifecycleNewTracker)
{
-MetadataCollector collector = new
MetadataCollector(metadata.comparator).sstableLevel(sstableLevel);
-return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, lifecycleNewTracker);
+MetadataCollector collector = new
MetadataCollector(metadata().comparator).sstableLevel(sstableLevel);
- return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
pendingRepair, isTransient, collector, header, txn);
++return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
pendingRepair, isTransient, collector, header, lifecycleNewTracker);
}
-
[jira] [Commented] (CASSANDRA-14554) Streaming needs to synchronise access to LifecycleTransaction
[
https://issues.apache.org/jira/browse/CASSANDRA-14554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714873#comment-16714873
]
Benedict commented on CASSANDRA-14554:
--
Thanks [~Stefania]. I've committed to
[84ffcb82a74667b957201f2cdae2d6b308956549|https://github.com/apache/cassandra/commit/84ffcb82a74667b957201f2cdae2d6b308956549]
on 3.0 and merged upwards.
> Streaming needs to synchronise access to LifecycleTransaction
> -
>
> Key: CASSANDRA-14554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14554
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Dinesh Joshi
>Assignee: Stefania
>Priority: Major
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
> When LifecycleTransaction is used in a multi-threaded context, we encounter
> this exception -
> {quote}java.util.ConcurrentModificationException: null
> at
> java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:719)
> at java.util.LinkedHashMap$LinkedKeyIterator.next(LinkedHashMap.java:742)
> at java.lang.Iterable.forEach(Iterable.java:74)
> at
> org.apache.cassandra.db.lifecycle.LogReplicaSet.maybeCreateReplica(LogReplicaSet.java:78)
> at org.apache.cassandra.db.lifecycle.LogFile.makeRecord(LogFile.java:320)
> at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:285)
> at
> org.apache.cassandra.db.lifecycle.LogTransaction.trackNew(LogTransaction.java:136)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.trackNew(LifecycleTransaction.java:529)
> {quote}
> During streaming we create a reference to a {{LifeCycleTransaction}} and
> share it between threads -
> [https://github.com/apache/cassandra/blob/5cc68a87359dd02412bdb70a52dfcd718d44a5ba/src/java/org/apache/cassandra/db/streaming/CassandraStreamReader.java#L156]
> This is used in a multi-threaded context inside {{CassandraIncomingFile}}
> which is an {{IncomingStreamMessage}}. This is being deserialized in parallel.
> {{LifecycleTransaction}} is not meant to be used in a multi-threaded context
> and this leads to streaming failures due to object sharing. On trunk, this
> object is shared across all threads that transfer sstables in parallel for
> the given {{TableId}} in a {{StreamSession}}. There are two options to solve
> this - make {{LifecycleTransaction}} and the associated objects thread safe,
> scope the transaction to a single {{CassandraIncomingFile}}. The consequences
> of the latter option is that if we experience streaming failure we may have
> redundant SSTables on disk. This is ok as compaction should clean this up. A
> third option is we synchronize access in the streaming infrastructure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[05/11] cassandra git commit: Streaming needs to synchronise access to LifecycleTransaction
Streaming needs to synchronise access to LifecycleTransaction
patch by Stefania Alborghetti and Benedict; reviewed by Robert Stupp for
CASSANDRA-14554
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84ffcb82
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84ffcb82
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84ffcb82
Branch: refs/heads/cassandra-3.11
Commit: 84ffcb82a74667b957201f2cdae2d6b308956549
Parents: bbf7dac
Author: Stefania Alborghetti
Authored: Wed Nov 7 14:07:12 2018 +0800
Committer: Benedict Elliott Smith
Committed: Mon Dec 10 15:00:00 2018 +
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 8 ++--
.../compaction/AbstractCompactionStrategy.java | 6 +--
.../compaction/CompactionStrategyManager.java | 8 ++--
.../db/lifecycle/LifecycleNewTracker.java | 47
.../db/lifecycle/LifecycleTransaction.java | 7 ++-
.../apache/cassandra/db/lifecycle/LogFile.java | 24 --
.../cassandra/db/lifecycle/LogTransaction.java | 2 +-
.../io/sstable/SimpleSSTableMultiWriter.java| 6 +--
.../io/sstable/format/SSTableWriter.java| 24 +-
.../io/sstable/format/big/BigFormat.java| 6 +--
.../io/sstable/format/big/BigTableWriter.java | 6 +--
.../cassandra/streaming/StreamReader.java | 6 +--
.../cassandra/streaming/StreamReceiveTask.java | 37 +--
.../cassandra/streaming/StreamSession.java | 4 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 9 ++--
16 files changed, 140 insertions(+), 61 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f9b59df..01d4789 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.18
+ * Streaming needs to synchronise access to LifecycleTransaction
(CASSANDRA-14554)
* Fix cassandra-stress write hang with default options (CASSANDRA-14616)
* Differentiate between slices and RTs when decoding legacy bounds
(CASSANDRA-14919)
* CommitLogReplayer.handleReplayError should print stack traces
(CASSANDRA-14589)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 4c7bc46..c455c4c 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -475,15 +475,15 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
return directories;
}
-public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleTransaction txn)
+public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleNewTracker lifecycleNewTracker)
{
MetadataCollector collector = new
MetadataCollector(metadata.comparator).sstableLevel(sstableLevel);
-return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, txn);
+return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, lifecycleNewTracker);
}
-public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleTransaction txn)
+public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleNewTracker lifecycleNewTracker)
{
-return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, txn);
+return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, lifecycleNewTracker);
}
public boolean supportsEarlyOpen()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
index a80a6f4..9f07691 100644
---
[03/11] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0918ba01
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0918ba01
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0918ba01
Branch: refs/heads/cassandra-3.11
Commit: 0918ba01176392deadf8655a61dad44979a49ee5
Parents: 6a449b8 3539a07
Author: Benedict Elliott Smith
Authored: Fri Dec 7 12:27:08 2018 +
Committer: Benedict Elliott Smith
Committed: Fri Dec 7 12:27:08 2018 +
--
.../apache/cassandra/db/ColumnFamilyStore.java | 8 ++--
.../compaction/AbstractCompactionStrategy.java | 5 ++-
.../compaction/CompactionStrategyManager.java | 14 +++---
.../db/lifecycle/LifecycleNewTracker.java | 47
.../db/lifecycle/LifecycleTransaction.java | 7 ++-
.../apache/cassandra/db/lifecycle/LogFile.java | 24 --
.../cassandra/db/lifecycle/LogTransaction.java | 2 +-
.../io/sstable/SimpleSSTableMultiWriter.java| 16 +++
.../sstable/format/RangeAwareSSTableWriter.java | 12 ++---
.../io/sstable/format/SSTableWriter.java| 24 +-
.../io/sstable/format/big/BigFormat.java| 6 +--
.../io/sstable/format/big/BigTableWriter.java | 6 +--
.../cassandra/streaming/StreamReader.java | 6 +--
.../cassandra/streaming/StreamReceiveTask.java | 35 ++-
.../cassandra/streaming/StreamSession.java | 4 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 5 ++-
16 files changed, 150 insertions(+), 71 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0918ba01/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 13f0280,c455c4c..700c1cc
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -507,15 -475,15 +507,15 @@@ public class ColumnFamilyStore implemen
return directories;
}
- public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleTransaction txn)
+ public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleNewTracker lifecycleNewTracker)
{
MetadataCollector collector = new
MetadataCollector(metadata.comparator).sstableLevel(sstableLevel);
- return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, txn);
+ return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, lifecycleNewTracker);
}
- public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleTransaction txn)
+ public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleNewTracker lifecycleNewTracker)
{
- return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, indexManager.listIndexes(), txn);
-return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, lifecycleNewTracker);
++return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, indexManager.listIndexes(),
lifecycleNewTracker);
}
public boolean supportsEarlyOpen()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0918ba01/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
--
diff --cc
src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
index 8454147,9f07691..3d7800d
---
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
@@@ -26,7 -26,8 +26,8 @@@ import com.google.common.collect.Iterab
import org.apache.cassandra.db.Directories;
import org.apache.cassandra.db.SerializationHeader;
+import org.apache.cassandra.index.Index;
+ import org.apache.cassandra.db.lifecycle.LifecycleNewTracker;
import org.apache.cassandra.io.sstable.Descriptor;
import org.apache.cassandra.io.sstable.SSTableMultiWriter;
import org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter;
@@@ -571,20 -514,9
[08/11] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/732c43b0 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/732c43b0 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/732c43b0 Branch: refs/heads/trunk Commit: 732c43b0f2dbc085c6e1da6498966fd656f32c2f Parents: 0918ba0 84ffcb8 Author: Benedict Elliott Smith Authored: Mon Dec 10 15:03:23 2018 + Committer: Benedict Elliott Smith Committed: Mon Dec 10 15:03:23 2018 + -- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/732c43b0/CHANGES.txt -- diff --cc CHANGES.txt index 369d96e,01d4789..20da1fa --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,9 -1,8 +1,10 @@@ -3.0.18 +3.11.4 + * Make stop-server.bat wait for Cassandra to terminate (CASSANDRA-14829) + * Correct sstable sorting for garbagecollect and levelled compaction (CASSANDRA-14870) +Merged from 3.0: + * Streaming needs to synchronise access to LifecycleTransaction (CASSANDRA-14554) * Fix cassandra-stress write hang with default options (CASSANDRA-14616) * Differentiate between slices and RTs when decoding legacy bounds (CASSANDRA-14919) - * CommitLogReplayer.handleReplayError should print stack traces (CASSANDRA-14589) * Netty epoll IOExceptions caused by unclean client disconnects being logged at INFO (CASSANDRA-14909) * Unfiltered.isEmpty conflicts with Row extends AbstractCollection.isEmpty (CASSANDRA-14588) * RangeTombstoneList doesn't properly clean up mergeable or superseded rts in some cases (CASSANDRA-14894) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[06/11] cassandra git commit: Streaming needs to synchronise access to LifecycleTransaction
Streaming needs to synchronise access to LifecycleTransaction
patch by Stefania Alborghetti and Benedict; reviewed by Robert Stupp for
CASSANDRA-14554
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84ffcb82
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84ffcb82
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84ffcb82
Branch: refs/heads/cassandra-3.0
Commit: 84ffcb82a74667b957201f2cdae2d6b308956549
Parents: bbf7dac
Author: Stefania Alborghetti
Authored: Wed Nov 7 14:07:12 2018 +0800
Committer: Benedict Elliott Smith
Committed: Mon Dec 10 15:00:00 2018 +
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 8 ++--
.../compaction/AbstractCompactionStrategy.java | 6 +--
.../compaction/CompactionStrategyManager.java | 8 ++--
.../db/lifecycle/LifecycleNewTracker.java | 47
.../db/lifecycle/LifecycleTransaction.java | 7 ++-
.../apache/cassandra/db/lifecycle/LogFile.java | 24 --
.../cassandra/db/lifecycle/LogTransaction.java | 2 +-
.../io/sstable/SimpleSSTableMultiWriter.java| 6 +--
.../io/sstable/format/SSTableWriter.java| 24 +-
.../io/sstable/format/big/BigFormat.java| 6 +--
.../io/sstable/format/big/BigTableWriter.java | 6 +--
.../cassandra/streaming/StreamReader.java | 6 +--
.../cassandra/streaming/StreamReceiveTask.java | 37 +--
.../cassandra/streaming/StreamSession.java | 4 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 9 ++--
16 files changed, 140 insertions(+), 61 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f9b59df..01d4789 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.18
+ * Streaming needs to synchronise access to LifecycleTransaction
(CASSANDRA-14554)
* Fix cassandra-stress write hang with default options (CASSANDRA-14616)
* Differentiate between slices and RTs when decoding legacy bounds
(CASSANDRA-14919)
* CommitLogReplayer.handleReplayError should print stack traces
(CASSANDRA-14589)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 4c7bc46..c455c4c 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -475,15 +475,15 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
return directories;
}
-public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleTransaction txn)
+public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleNewTracker lifecycleNewTracker)
{
MetadataCollector collector = new
MetadataCollector(metadata.comparator).sstableLevel(sstableLevel);
-return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, txn);
+return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, lifecycleNewTracker);
}
-public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleTransaction txn)
+public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleNewTracker lifecycleNewTracker)
{
-return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, txn);
+return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, lifecycleNewTracker);
}
public boolean supportsEarlyOpen()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
index a80a6f4..9f07691 100644
---
[01/11] cassandra git commit: Prevent concurrent access to lifecycle txn when streaming
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 bbf7dac87 -> 84ffcb82a
refs/heads/cassandra-3.11 6a449b88d -> 732c43b0f
refs/heads/trunk 4dd7faa75 -> 33eada06a
Prevent concurrent access to lifecycle txn when streaming
patch by Stefania Alborghetti; reviewed by Robert Stupp for CASSANDRA-14554
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3539a078
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3539a078
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3539a078
Branch: refs/heads/cassandra-3.11
Commit: 3539a07884d0a94c1432d7dda0ec17676195cc53
Parents: bbf7dac
Author: Stefania Alborghetti
Authored: Wed Nov 7 14:07:12 2018 +0800
Committer: Benedict Elliott Smith
Committed: Fri Dec 7 12:18:05 2018 +
--
.../apache/cassandra/db/ColumnFamilyStore.java | 8 ++--
.../compaction/AbstractCompactionStrategy.java | 6 +--
.../compaction/CompactionStrategyManager.java | 8 ++--
.../db/lifecycle/LifecycleNewTracker.java | 47
.../db/lifecycle/LifecycleTransaction.java | 7 ++-
.../apache/cassandra/db/lifecycle/LogFile.java | 24 --
.../cassandra/db/lifecycle/LogTransaction.java | 2 +-
.../io/sstable/SimpleSSTableMultiWriter.java| 6 +--
.../io/sstable/format/SSTableWriter.java| 24 +-
.../io/sstable/format/big/BigFormat.java| 6 +--
.../io/sstable/format/big/BigTableWriter.java | 6 +--
.../cassandra/streaming/StreamReader.java | 6 +--
.../cassandra/streaming/StreamReceiveTask.java | 37 +--
.../cassandra/streaming/StreamSession.java | 4 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 9 ++--
15 files changed, 139 insertions(+), 61 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3539a078/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 4c7bc46..c455c4c 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -475,15 +475,15 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
return directories;
}
-public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleTransaction txn)
+public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleNewTracker lifecycleNewTracker)
{
MetadataCollector collector = new
MetadataCollector(metadata.comparator).sstableLevel(sstableLevel);
-return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, txn);
+return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, lifecycleNewTracker);
}
-public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleTransaction txn)
+public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleNewTracker lifecycleNewTracker)
{
-return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, txn);
+return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, lifecycleNewTracker);
}
public boolean supportsEarlyOpen()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3539a078/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
index a80a6f4..9f07691 100644
---
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
@@ -27,7 +27,7 @@ import com.google.common.util.concurrent.RateLimiter;
import org.apache.cassandra.db.Directories;
import org.apache.cassandra.db.SerializationHeader;
-import org.apache.cassandra.db.lifecycle.SSTableSet;
+import org.apache.cassandra.db.lifecycle.LifecycleNewTracker;
import org.apache.cassandra.io.sstable.Descriptor;
import
[02/11] cassandra git commit: Prevent concurrent access to lifecycle txn when streaming
Prevent concurrent access to lifecycle txn when streaming
patch by Stefania Alborghetti; reviewed by Robert Stupp for CASSANDRA-14554
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3539a078
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3539a078
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3539a078
Branch: refs/heads/trunk
Commit: 3539a07884d0a94c1432d7dda0ec17676195cc53
Parents: bbf7dac
Author: Stefania Alborghetti
Authored: Wed Nov 7 14:07:12 2018 +0800
Committer: Benedict Elliott Smith
Committed: Fri Dec 7 12:18:05 2018 +
--
.../apache/cassandra/db/ColumnFamilyStore.java | 8 ++--
.../compaction/AbstractCompactionStrategy.java | 6 +--
.../compaction/CompactionStrategyManager.java | 8 ++--
.../db/lifecycle/LifecycleNewTracker.java | 47
.../db/lifecycle/LifecycleTransaction.java | 7 ++-
.../apache/cassandra/db/lifecycle/LogFile.java | 24 --
.../cassandra/db/lifecycle/LogTransaction.java | 2 +-
.../io/sstable/SimpleSSTableMultiWriter.java| 6 +--
.../io/sstable/format/SSTableWriter.java| 24 +-
.../io/sstable/format/big/BigFormat.java| 6 +--
.../io/sstable/format/big/BigTableWriter.java | 6 +--
.../cassandra/streaming/StreamReader.java | 6 +--
.../cassandra/streaming/StreamReceiveTask.java | 37 +--
.../cassandra/streaming/StreamSession.java | 4 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 9 ++--
15 files changed, 139 insertions(+), 61 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3539a078/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 4c7bc46..c455c4c 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -475,15 +475,15 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
return directories;
}
-public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleTransaction txn)
+public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleNewTracker lifecycleNewTracker)
{
MetadataCollector collector = new
MetadataCollector(metadata.comparator).sstableLevel(sstableLevel);
-return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, txn);
+return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, lifecycleNewTracker);
}
-public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleTransaction txn)
+public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleNewTracker lifecycleNewTracker)
{
-return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, txn);
+return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, lifecycleNewTracker);
}
public boolean supportsEarlyOpen()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3539a078/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
index a80a6f4..9f07691 100644
---
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
@@ -27,7 +27,7 @@ import com.google.common.util.concurrent.RateLimiter;
import org.apache.cassandra.db.Directories;
import org.apache.cassandra.db.SerializationHeader;
-import org.apache.cassandra.db.lifecycle.SSTableSet;
+import org.apache.cassandra.db.lifecycle.LifecycleNewTracker;
import org.apache.cassandra.io.sstable.Descriptor;
import org.apache.cassandra.io.sstable.SSTableMultiWriter;
import org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter;
@@ -514,9 +514,9 @@ public abstract class AbstractCompactionStrategy
return
[10/11] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
http://git-wip-us.apache.org/repos/asf/cassandra/blob/33eada06/src/java/org/apache/cassandra/io/sstable/format/SSTableWriter.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/format/SSTableWriter.java
index cca59cf,e320f30..f54bc03
--- a/src/java/org/apache/cassandra/io/sstable/format/SSTableWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableWriter.java
@@@ -104,10 -96,10 +104,10 @@@ public abstract class SSTableWriter ext
MetadataCollector metadataCollector,
SerializationHeader header,
Collection indexes,
-LifecycleTransaction txn)
+LifecycleNewTracker
lifecycleNewTracker)
{
Factory writerFactory = descriptor.getFormat().getWriterFactory();
- return writerFactory.open(descriptor, keyCount, repairedAt,
pendingRepair, isTransient, metadata, metadataCollector, header,
observers(descriptor, indexes, txn.opType()), txn);
-return writerFactory.open(descriptor, keyCount, repairedAt, metadata,
metadataCollector, header, observers(descriptor, indexes,
lifecycleNewTracker.opType()), lifecycleNewTracker);
++return writerFactory.open(descriptor, keyCount, repairedAt,
pendingRepair, isTransient, metadata, metadataCollector, header,
observers(descriptor, indexes, lifecycleNewTracker.opType()),
lifecycleNewTracker);
}
public static SSTableWriter create(Descriptor descriptor,
@@@ -118,13 -108,13 +118,13 @@@
int sstableLevel,
SerializationHeader header,
Collection indexes,
-LifecycleTransaction txn)
+LifecycleNewTracker
lifecycleNewTracker)
{
-CFMetaData metadata = Schema.instance.getCFMetaData(descriptor);
-return create(metadata, descriptor, keyCount, repairedAt,
sstableLevel, header, indexes, lifecycleNewTracker);
+TableMetadataRef metadata =
Schema.instance.getTableMetadataRef(descriptor);
- return create(metadata, descriptor, keyCount, repairedAt,
pendingRepair, isTransient, sstableLevel, header, indexes, txn);
++return create(metadata, descriptor, keyCount, repairedAt,
pendingRepair, isTransient, sstableLevel, header, indexes, lifecycleNewTracker);
}
-public static SSTableWriter create(CFMetaData metadata,
+public static SSTableWriter create(TableMetadataRef metadata,
Descriptor descriptor,
long keyCount,
long repairedAt,
@@@ -133,26 -121,36 +133,26 @@@
int sstableLevel,
SerializationHeader header,
Collection indexes,
-LifecycleTransaction txn)
+LifecycleNewTracker
lifecycleNewTracker)
{
-MetadataCollector collector = new
MetadataCollector(metadata.comparator).sstableLevel(sstableLevel);
-return create(descriptor, keyCount, repairedAt, metadata, collector,
header, indexes, lifecycleNewTracker);
-}
-
-public static SSTableWriter create(String filename,
- long keyCount,
- long repairedAt,
- int sstableLevel,
- SerializationHeader header,
- Collection indexes,
- LifecycleNewTracker
lifecycleNewTracker)
-{
-return create(Descriptor.fromFilename(filename), keyCount,
repairedAt, sstableLevel, header, indexes, lifecycleNewTracker);
+MetadataCollector collector = new
MetadataCollector(metadata.get().comparator).sstableLevel(sstableLevel);
- return create(descriptor, keyCount, repairedAt, pendingRepair,
isTransient, metadata, collector, header, indexes, txn);
++return create(descriptor, keyCount, repairedAt, pendingRepair,
isTransient, metadata, collector, header, indexes, lifecycleNewTracker);
}
@VisibleForTesting
-public static SSTableWriter create(String filename,
+public static SSTableWriter create(Descriptor descriptor,
long keyCount,
long repairedAt,
+ UUID pendingRepair,
+ boolean isTransient,
SerializationHeader header,
[07/11] cassandra git commit: Streaming needs to synchronise access to LifecycleTransaction
Streaming needs to synchronise access to LifecycleTransaction
patch by Stefania Alborghetti and Benedict; reviewed by Robert Stupp for
CASSANDRA-14554
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84ffcb82
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84ffcb82
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84ffcb82
Branch: refs/heads/trunk
Commit: 84ffcb82a74667b957201f2cdae2d6b308956549
Parents: bbf7dac
Author: Stefania Alborghetti
Authored: Wed Nov 7 14:07:12 2018 +0800
Committer: Benedict Elliott Smith
Committed: Mon Dec 10 15:00:00 2018 +
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 8 ++--
.../compaction/AbstractCompactionStrategy.java | 6 +--
.../compaction/CompactionStrategyManager.java | 8 ++--
.../db/lifecycle/LifecycleNewTracker.java | 47
.../db/lifecycle/LifecycleTransaction.java | 7 ++-
.../apache/cassandra/db/lifecycle/LogFile.java | 24 --
.../cassandra/db/lifecycle/LogTransaction.java | 2 +-
.../io/sstable/SimpleSSTableMultiWriter.java| 6 +--
.../io/sstable/format/SSTableWriter.java| 24 +-
.../io/sstable/format/big/BigFormat.java| 6 +--
.../io/sstable/format/big/BigTableWriter.java | 6 +--
.../cassandra/streaming/StreamReader.java | 6 +--
.../cassandra/streaming/StreamReceiveTask.java | 37 +--
.../cassandra/streaming/StreamSession.java | 4 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 9 ++--
16 files changed, 140 insertions(+), 61 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f9b59df..01d4789 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.18
+ * Streaming needs to synchronise access to LifecycleTransaction
(CASSANDRA-14554)
* Fix cassandra-stress write hang with default options (CASSANDRA-14616)
* Differentiate between slices and RTs when decoding legacy bounds
(CASSANDRA-14919)
* CommitLogReplayer.handleReplayError should print stack traces
(CASSANDRA-14589)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 4c7bc46..c455c4c 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -475,15 +475,15 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
return directories;
}
-public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleTransaction txn)
+public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleNewTracker lifecycleNewTracker)
{
MetadataCollector collector = new
MetadataCollector(metadata.comparator).sstableLevel(sstableLevel);
-return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, txn);
+return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, lifecycleNewTracker);
}
-public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleTransaction txn)
+public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleNewTracker lifecycleNewTracker)
{
-return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, txn);
+return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, lifecycleNewTracker);
}
public boolean supportsEarlyOpen()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
index a80a6f4..9f07691 100644
---
[09/11] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/732c43b0 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/732c43b0 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/732c43b0 Branch: refs/heads/cassandra-3.11 Commit: 732c43b0f2dbc085c6e1da6498966fd656f32c2f Parents: 0918ba0 84ffcb8 Author: Benedict Elliott Smith Authored: Mon Dec 10 15:03:23 2018 + Committer: Benedict Elliott Smith Committed: Mon Dec 10 15:03:23 2018 + -- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/732c43b0/CHANGES.txt -- diff --cc CHANGES.txt index 369d96e,01d4789..20da1fa --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,9 -1,8 +1,10 @@@ -3.0.18 +3.11.4 + * Make stop-server.bat wait for Cassandra to terminate (CASSANDRA-14829) + * Correct sstable sorting for garbagecollect and levelled compaction (CASSANDRA-14870) +Merged from 3.0: + * Streaming needs to synchronise access to LifecycleTransaction (CASSANDRA-14554) * Fix cassandra-stress write hang with default options (CASSANDRA-14616) * Differentiate between slices and RTs when decoding legacy bounds (CASSANDRA-14919) - * CommitLogReplayer.handleReplayError should print stack traces (CASSANDRA-14589) * Netty epoll IOExceptions caused by unclean client disconnects being logged at INFO (CASSANDRA-14909) * Unfiltered.isEmpty conflicts with Row extends AbstractCollection.isEmpty (CASSANDRA-14588) * RangeTombstoneList doesn't properly clean up mergeable or superseded rts in some cases (CASSANDRA-14894) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[04/11] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0918ba01
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0918ba01
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0918ba01
Branch: refs/heads/trunk
Commit: 0918ba01176392deadf8655a61dad44979a49ee5
Parents: 6a449b8 3539a07
Author: Benedict Elliott Smith
Authored: Fri Dec 7 12:27:08 2018 +
Committer: Benedict Elliott Smith
Committed: Fri Dec 7 12:27:08 2018 +
--
.../apache/cassandra/db/ColumnFamilyStore.java | 8 ++--
.../compaction/AbstractCompactionStrategy.java | 5 ++-
.../compaction/CompactionStrategyManager.java | 14 +++---
.../db/lifecycle/LifecycleNewTracker.java | 47
.../db/lifecycle/LifecycleTransaction.java | 7 ++-
.../apache/cassandra/db/lifecycle/LogFile.java | 24 --
.../cassandra/db/lifecycle/LogTransaction.java | 2 +-
.../io/sstable/SimpleSSTableMultiWriter.java| 16 +++
.../sstable/format/RangeAwareSSTableWriter.java | 12 ++---
.../io/sstable/format/SSTableWriter.java| 24 +-
.../io/sstable/format/big/BigFormat.java| 6 +--
.../io/sstable/format/big/BigTableWriter.java | 6 +--
.../cassandra/streaming/StreamReader.java | 6 +--
.../cassandra/streaming/StreamReceiveTask.java | 35 ++-
.../cassandra/streaming/StreamSession.java | 4 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 5 ++-
16 files changed, 150 insertions(+), 71 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0918ba01/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 13f0280,c455c4c..700c1cc
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -507,15 -475,15 +507,15 @@@ public class ColumnFamilyStore implemen
return directories;
}
- public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleTransaction txn)
+ public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, int sstableLevel, SerializationHeader header,
LifecycleNewTracker lifecycleNewTracker)
{
MetadataCollector collector = new
MetadataCollector(metadata.comparator).sstableLevel(sstableLevel);
- return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, txn);
+ return createSSTableMultiWriter(descriptor, keyCount, repairedAt,
collector, header, lifecycleNewTracker);
}
- public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleTransaction txn)
+ public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
long keyCount, long repairedAt, MetadataCollector metadataCollector,
SerializationHeader header, LifecycleNewTracker lifecycleNewTracker)
{
- return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, indexManager.listIndexes(), txn);
-return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, lifecycleNewTracker);
++return
getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount,
repairedAt, metadataCollector, header, indexManager.listIndexes(),
lifecycleNewTracker);
}
public boolean supportsEarlyOpen()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0918ba01/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
--
diff --cc
src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
index 8454147,9f07691..3d7800d
---
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
@@@ -26,7 -26,8 +26,8 @@@ import com.google.common.collect.Iterab
import org.apache.cassandra.db.Directories;
import org.apache.cassandra.db.SerializationHeader;
+import org.apache.cassandra.index.Index;
+ import org.apache.cassandra.db.lifecycle.LifecycleNewTracker;
import org.apache.cassandra.io.sstable.Descriptor;
import org.apache.cassandra.io.sstable.SSTableMultiWriter;
import org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter;
@@@ -571,20 -514,9 +572,20 @@@
[jira] [Updated] (CASSANDRA-14554) Streaming needs to synchronise access to LifecycleTransaction
[
https://issues.apache.org/jira/browse/CASSANDRA-14554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict updated CASSANDRA-14554:
-
Summary: Streaming needs to synchronise access to LifecycleTransaction
(was: LifecycleTransaction encounters ConcurrentModificationException when used
in multi-threaded context)
> Streaming needs to synchronise access to LifecycleTransaction
> -
>
> Key: CASSANDRA-14554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14554
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Dinesh Joshi
>Assignee: Stefania
>Priority: Major
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
> When LifecycleTransaction is used in a multi-threaded context, we encounter
> this exception -
> {quote}java.util.ConcurrentModificationException: null
> at
> java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:719)
> at java.util.LinkedHashMap$LinkedKeyIterator.next(LinkedHashMap.java:742)
> at java.lang.Iterable.forEach(Iterable.java:74)
> at
> org.apache.cassandra.db.lifecycle.LogReplicaSet.maybeCreateReplica(LogReplicaSet.java:78)
> at org.apache.cassandra.db.lifecycle.LogFile.makeRecord(LogFile.java:320)
> at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:285)
> at
> org.apache.cassandra.db.lifecycle.LogTransaction.trackNew(LogTransaction.java:136)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.trackNew(LifecycleTransaction.java:529)
> {quote}
> During streaming we create a reference to a {{LifeCycleTransaction}} and
> share it between threads -
> [https://github.com/apache/cassandra/blob/5cc68a87359dd02412bdb70a52dfcd718d44a5ba/src/java/org/apache/cassandra/db/streaming/CassandraStreamReader.java#L156]
> This is used in a multi-threaded context inside {{CassandraIncomingFile}}
> which is an {{IncomingStreamMessage}}. This is being deserialized in parallel.
> {{LifecycleTransaction}} is not meant to be used in a multi-threaded context
> and this leads to streaming failures due to object sharing. On trunk, this
> object is shared across all threads that transfer sstables in parallel for
> the given {{TableId}} in a {{StreamSession}}. There are two options to solve
> this - make {{LifecycleTransaction}} and the associated objects thread safe,
> scope the transaction to a single {{CassandraIncomingFile}}. The consequences
> of the latter option is that if we experience streaming failure we may have
> redundant SSTables on disk. This is ok as compaction should clean this up. A
> third option is we synchronize access in the streaming infrastructure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14871) Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser
[
https://issues.apache.org/jira/browse/CASSANDRA-14871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714681#comment-16714681
]
Robert Stupp commented on CASSANDRA-14871:
--
I'm fine with using {{Verify}}, but it should include the string that yielded
{{null}}. Like {{Verify.verify(type != null, "Parsing %s yielded null, which is
a bug", type)}} and move that after the assignment of {{type}}. That way we
don't need the {{cache.get(str)}} in the synchronized block, because {{type}}
is then guaranteed to be non-null.
{code:java}
synchronized (TypeParser.class)
{
if (!cache.containsKey(str))
{
ImmutableMap.Builder> builder =
ImmutableMap.builder();
builder.putAll(cache).put(str, type);
cache = builder.build();
}
AbstractType rtype = cache.get(str);
Verify.verify(rtype != null);
return rtype;
}
{code}
> Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser
> ---
>
> Key: CASSANDRA-14871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14871
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Critical
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
> There are a couple of places in the code base that do not respect that
> j.u.HashMap + related classes are not thread safe and some parts rely on
> internals of the implementation of HM, which can change.
> We have observed failures like {{NullPointerException}} and
> {{ConcurrentModificationException}} as well as wrong behavior.
> Affected areas in the code base:
> * {{SizeTieredCompactionStrategy}}
> * {{DateTieredCompactionStrategy}}
> * {{TimeWindowCompactionStrategy}}
> * {{TokenMetadata.Topology}}
> * {{TypeParser}}
> * streaming / concurrent access to {{LifecycleTransaction}} (handled in
> CASSANDRA-14554)
> While the patches for the compaction strategies + {{TypeParser}} are pretty
> straight forward, the patch for {{TokenMetadata.Topology}} requires it to be
> made immutable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14925) DecimalSerializer.toString() can be used as OOM attack
[
https://issues.apache.org/jira/browse/CASSANDRA-14925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714659#comment-16714659
]
ZhaoYang commented on CASSANDRA-14925:
--
{quote}Can't we just use BigDecimal.toString() all the time as save ourselves
the trouble of adding yet one more runtime parameter that no user will probably
ever modify?{quote}
Make sense.. updated the patch to use `toString()` only.
|patch|circle-ci|
|[3.0|https://github.com/jasonstack/cassandra/commits/decimal-tostring-3.0]|[unit|https://circleci.com/gh/jasonstack/cassandra/747?utm_campaign=vcs-integration-link_medium=referral_source=github-build-link]|
|[3.11|https://github.com/jasonstack/cassandra/commits/decimal-tostring-3.11]|[unit|https://circleci.com/gh/jasonstack/cassandra/752?utm_campaign=vcs-integration-link_medium=referral_source=github-build-link]|
|[trunk|https://github.com/jasonstack/cassandra/commits/decimal-tostring-trunk]|[unit|https://circleci.com/gh/jasonstack/cassandra/751?utm_campaign=vcs-integration-link_medium=referral_source=github-build-link]|
> DecimalSerializer.toString() can be used as OOM attack
> ---
>
> Key: CASSANDRA-14925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14925
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Minor
>
> Currently, in {{DecimalSerializer.toString(value)}}, it uses
> {{BigDecimal.toPlainString()}} which generates huge string for large scale
> values.
>
> {code:java}
> BigDecimal d = new BigDecimal("1e-" + (Integer.MAX_VALUE - 6));
> d.toPlainString(); // oom{code}
>
> Propose to use {{BigDecimal.toString()}} when scale is larger than 100 which
> is configurable via {{-Dcassandra.decimal.maxscaleforstring}}
>
> | patch | circle-ci |
> | [3.0|https://github.com/jasonstack/cassandra/commits/decimal-tostring-3.0]
> |
> [unit|https://circleci.com/gh/jasonstack/cassandra/747?utm_campaign=vcs-integration-link_medium=referral_source=github-build-link]
> |
> The code should apply cleanly to 3.0+.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14482) ZSTD Compressor support in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714590#comment-16714590 ] Benedict commented on CASSANDRA-14482: -- [~jolynch]: I know we've talked about this before, but once 4.0 freeze is over I'd be happy to help you get this dictionary approach merged into mainline (and help you figure out any issues with your interactions with early-open - presumably the issue is with sharing ownership of the new dictionary). I'm not sure it would strictly be necessary to wait until 5.0 for this, as while it certainly breaks _my_ interpretation of the freeze, it's probably not going to be a _dramatic_ change. It could be quite localised, and might be acceptable for a patch release, depending on how we approach our future release processes. > ZSTD Compressor support in Cassandra > > > Key: CASSANDRA-14482 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14482 > Project: Cassandra > Issue Type: New Feature > Components: Compression, Libraries >Reporter: Sushma A Devendrappa >Assignee: Sushma A Devendrappa >Priority: Major > Labels: performance, pull-request-available > Fix For: 4.x > > Time Spent: 1h 10m > Remaining Estimate: 0h > > ZStandard has a great speed and compression ratio tradeoff. > ZStandard is open source compression from Facebook. > More about ZSTD > [https://github.com/facebook/zstd] > https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/ > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14925) DecimalSerializer.toString() can be used as OOM attack
[
https://issues.apache.org/jira/browse/CASSANDRA-14925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714525#comment-16714525
]
Sylvain Lebresne commented on CASSANDRA-14925:
--
Can't we just use {{BigDecimal.toString()}} all the time as save ourselves the
trouble of adding yet one more runtime parameter that no user will probably
ever modify?
> DecimalSerializer.toString() can be used as OOM attack
> ---
>
> Key: CASSANDRA-14925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14925
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Minor
>
> Currently, in {{DecimalSerializer.toString(value)}}, it uses
> {{BigDecimal.toPlainString()}} which generates huge string for large scale
> values.
>
> {code:java}
> BigDecimal d = new BigDecimal("1e-" + (Integer.MAX_VALUE - 6));
> d.toPlainString(); // oom{code}
>
> Propose to use {{BigDecimal.toString()}} when scale is larger than 100 which
> is configurable via {{-Dcassandra.decimal.maxscaleforstring}}
>
> | patch | circle-ci |
> | [3.0|https://github.com/jasonstack/cassandra/commits/decimal-tostring-3.0]
> |
> [unit|https://circleci.com/gh/jasonstack/cassandra/747?utm_campaign=vcs-integration-link_medium=referral_source=github-build-link]
> |
> The code should apply cleanly to 3.0+.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
