[jira] [Commented] (CASSANDRA-14917) Nodetool netstats displays incorrect information on streaming
[
https://issues.apache.org/jira/browse/CASSANDRA-14917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746880#comment-16746880
]
Dinesh Joshi commented on CASSANDRA-14917:
--
[~geagle] any chance you tried this on trunk?
> Nodetool netstats displays incorrect information on streaming
> -
>
> Key: CASSANDRA-14917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14917
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Igor Zubchenok
>Assignee: Dinesh Joshi
>Priority: Minor
>
> nodetool netstats command displays 'Sending 0 files' and non-zero 'bytes
> total' during node decommission.
> {code:java}
> Sending 0 files, 198933290919 bytes total. Already sent 0 files, 0 bytes total
> {code}
> Complete notetool netstats response
> {code:java}
> # nodetool netstats | grep -v 100%
> Mode: LEAVING
> Unbootstrap 67096090-f604-11e8--03532290668f
> /X.X.X.X
> Sending 8039 files, 174905861575 bytes total. Already sent 3147
> files, 39883605552 bytes total
>
> /var/lib/cassandra/data/app_tracks/tracks-bfd4a21086b711e7b6cbfd84ea9f1f78/mc-100884-big-Data.db
> 1866267759/8275615307 bytes(22%) sent to idx:0/5.9.54.111
> /X.X.X.X
> Sending 5475 files, 167399940563 bytes total. Already sent 2416
> files, 49417386715 bytes total
>
> /var/lib/cassandra/data/app_tracks/tracks-bd8310f086b711e7b6cbfd84ea9f1f78/mc-459374-big-Data.db
> 10192112/17529144 bytes(58%) sent to idx:0/5.9.59.101
> /X.X.X.X
> Sending 5843 files, 181124719187 bytes total. Already sent 2613
> files, 57001466824 bytes total
>
> /var/lib/cassandra/data/app_tracks/tracks-bd8310f086b711e7b6cbfd84ea9f1f78/mc-461873-big-Data.db
> 1439600/93308008 bytes(1%) sent to idx:0/46.4.31.97
> /X.X.X.X
> Sending 7303 files, 213976795121 bytes total. Already sent 1811
> files, 28956350780 bytes total
>
> /var/lib/cassandra/data/app_tracks/tracks-bfd4a21086b711e7b6cbfd84ea9f1f78/mc-100465-big-Data.db
> 3719051/37149976 bytes(10%) sent to idx:0/144.76.85.237
> /X.X.X.X
> Sending 7893 files, 204271020294 bytes total. Already sent 2397
> files, 36086600359 bytes total
>
> /var/lib/cassandra/data/app_tracks/tracks-bfd4a21086b711e7b6cbfd84ea9f1f78/mc-100197-big-Data.db
> 646314/8291439 bytes(7%) sent to idx:0/144.76.29.39
> /X.X.X.X
> Sending 0 files, 198933290919 bytes total. Already sent 0 files, 0
> bytes total
> /X.X.X.X
> Sending 6159 files, 187102577561 bytes total. Already sent 1526
> files, 26019708822 bytes total
>
> /var/lib/cassandra/data/app_tracks/tracks-bfd4a21086b711e7b6cbfd84ea9f1f78/mc-100685-big-Data.db
> 52211410/67213497 bytes(77%) sent to idx:0/144.76.85.182
> /X.X.X.X
> Sending 6534 files, 193239644509 bytes total. Already sent 2367
> files, 48105187871 bytes total
>
> /var/lib/cassandra/data/app_tracks/tracks-bfd4a21086b711e7b6cbfd84ea9f1f78/mc-100884-big-Data.db
> 5907583715/10068538729 bytes(58%) sent to idx:0/5.9.48.222
> /X.X.X.X
> Sending 5923 files, 213438389441 bytes total. Already sent 1663
> files, 32469866925 bytes total
> Read Repair Statistics:
> Attempted: 3537331
> Mismatch (Blocking): 2193
> Mismatch (Background): 4326
> Pool NameActive Pending Completed Dropped
> Large messages n/a18 87460 746
> Small messages n/a 8 9837560424 12410
> Gossip messages n/a 87666199 84717
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14991) SSL Cert Hot Reloading should check for sanity of the new keystore/truststore before loading it
[
https://issues.apache.org/jira/browse/CASSANDRA-14991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746872#comment-16746872
]
Dinesh Joshi commented on CASSANDRA-14991:
--
The latest run is clean. I had to rebase the branch on latest trunk.
> SSL Cert Hot Reloading should check for sanity of the new keystore/truststore
> before loading it
> ---
>
> Key: CASSANDRA-14991
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14991
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Encryption
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Major
> Labels: security
> Fix For: 4.0
>
>
> SSL Cert Hot Reloading assumes that the keystore & truststore are valid.
> However, a corrupt store or a password mismatch can cause Cassandra to fail
> accepting new connections as we throw away the old {{SslContext}}. This patch
> will ensure that we check the sanity of the certificates during startup and
> during hot reloading. This should protect against bad key/trust stores. As
> part of this PR, I have cleaned up the code a bit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14989) NullPointerException when SELECTing token() on only one part of a two-part partition key
[
https://issues.apache.org/jira/browse/CASSANDRA-14989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-14989:
-
Reviewers: Jon Meredith
> NullPointerException when SELECTing token() on only one part of a two-part
> partition key
>
>
> Key: CASSANDRA-14989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14989
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
> Environment: Using {{cqlsh 5.0.1}} on a Mac OS X host system with
> Cassandra 3.11.3 running via Docker for Mac from the official
> {{cassandra:3.11.3}} image.
>Reporter: Manuel Kießling
>Assignee: Dinesh Joshi
>Priority: Major
>
> I have the following schema:
> {code}
> CREATE TABLE query_tests.cart_snapshots (
> cart_id uuid,
> realm text,
> snapshot_id timeuuid,
> state text,
> PRIMARY KEY ((cart_id, realm), snapshot_id)
> ) WITH CLUSTERING ORDER BY (snapshot_id DESC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}
> In cqlsh, I try the following query:
> {code}select token(cart_id) from cart_snapshots ;{code}
> This results in cqlsh returning {{ServerError:
> java.lang.NullPointerException}}, and the following error in the server log:
> {code}
> DC1N1_1 | ERROR [Native-Transport-Requests-1] 2019-01-16 12:17:52,075
> QueryMessage.java:129 - Unexpected error during query
> DC1N1_1 | java.lang.NullPointerException: null
> DC1N1_1 | at
> org.apache.cassandra.db.marshal.CompositeType.build(CompositeType.java:356)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.db.marshal.CompositeType.build(CompositeType.java:349)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.config.CFMetaData.serializePartitionKey(CFMetaData.java:805)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.functions.TokenFct.execute(TokenFct.java:59)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.selection.ScalarFunctionSelector.getOutput(ScalarFunctionSelector.java:61)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.selection.Selection$SelectionWithProcessing$1.getOutputRow(Selection.java:666)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.selection.Selection$ResultSetBuilder.getOutputRow(Selection.java:492)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.selection.Selection$ResultSetBuilder.newRow(Selection.java:458)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.statements.SelectStatement.processPartition(SelectStatement.java:860)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:790)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:438)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:416)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:289)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:117)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:224)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:255)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:240)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
>
[jira] [Assigned] (CASSANDRA-14989) NullPointerException when SELECTing token() on only one part of a two-part partition key
[
https://issues.apache.org/jira/browse/CASSANDRA-14989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi reassigned CASSANDRA-14989:
Assignee: Dinesh Joshi
> NullPointerException when SELECTing token() on only one part of a two-part
> partition key
>
>
> Key: CASSANDRA-14989
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14989
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
> Environment: Using {{cqlsh 5.0.1}} on a Mac OS X host system with
> Cassandra 3.11.3 running via Docker for Mac from the official
> {{cassandra:3.11.3}} image.
>Reporter: Manuel Kießling
>Assignee: Dinesh Joshi
>Priority: Major
>
> I have the following schema:
> {code}
> CREATE TABLE query_tests.cart_snapshots (
> cart_id uuid,
> realm text,
> snapshot_id timeuuid,
> state text,
> PRIMARY KEY ((cart_id, realm), snapshot_id)
> ) WITH CLUSTERING ORDER BY (snapshot_id DESC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}
> In cqlsh, I try the following query:
> {code}select token(cart_id) from cart_snapshots ;{code}
> This results in cqlsh returning {{ServerError:
> java.lang.NullPointerException}}, and the following error in the server log:
> {code}
> DC1N1_1 | ERROR [Native-Transport-Requests-1] 2019-01-16 12:17:52,075
> QueryMessage.java:129 - Unexpected error during query
> DC1N1_1 | java.lang.NullPointerException: null
> DC1N1_1 | at
> org.apache.cassandra.db.marshal.CompositeType.build(CompositeType.java:356)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.db.marshal.CompositeType.build(CompositeType.java:349)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.config.CFMetaData.serializePartitionKey(CFMetaData.java:805)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.functions.TokenFct.execute(TokenFct.java:59)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.selection.ScalarFunctionSelector.getOutput(ScalarFunctionSelector.java:61)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.selection.Selection$SelectionWithProcessing$1.getOutputRow(Selection.java:666)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.selection.Selection$ResultSetBuilder.getOutputRow(Selection.java:492)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.selection.Selection$ResultSetBuilder.newRow(Selection.java:458)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.statements.SelectStatement.processPartition(SelectStatement.java:860)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:790)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:438)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:416)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:289)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:117)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:224)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:255)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:240)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> DC1N1_1 | at
>
[jira] [Commented] (CASSANDRA-13508) Make system.paxos table compaction strategy configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746698#comment-16746698
]
Blake Eggleston commented on CASSANDRA-13508:
-
Keyspace level lwts would be cool, but I think they've always been more of a
pony than a feature anyone was seriously considering implementing. Implicitly
expanding paxos granularity from table/primary_key into keyspace/primary_key
isn't a good idea imo, since you can introduce cross table contention that
users don't necessarily need or want. Also, assuming keyspace lwts does become
a thing, splitting system.paxos up per table wouldn't necessarily preclude
them.
> Make system.paxos table compaction strategy configurable
>
>
> Key: CASSANDRA-13508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13508
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Coordination
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Major
> Labels: LWT, core, paxos
> Fix For: 4.x
>
> Attachments: test11.png, test2.png
>
>
> The default compaction strategy for {{system.paxos}} table is LCS for
> performance reason: CASSANDRA-7753. But for CAS heavily used cluster, the
> system is busy with {{system.paxos}} compaction.
> As the data in {{paxos}} table are TTL'ed, TWCS might be a better fit. In our
> test, it significantly reduced the number of compaction without impacting the
> latency too much:
> !test11.png!
> The time window for TWCS is set to 2 minutes for the test.
> Here is the p99 latency impact:
> !test2.png!
> the yellow one is LCS, the purple one is TWCS. Average p99 has about 10%
> increase.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14991) SSL Cert Hot Reloading should check for sanity of the new keystore/truststore before loading it
[
https://issues.apache.org/jira/browse/CASSANDRA-14991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-14991:
-
Fix Version/s: 4.0
Status: Patch Available (was: Open)
> SSL Cert Hot Reloading should check for sanity of the new keystore/truststore
> before loading it
> ---
>
> Key: CASSANDRA-14991
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14991
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Encryption
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Major
> Labels: security
> Fix For: 4.0
>
>
> SSL Cert Hot Reloading assumes that the keystore & truststore are valid.
> However, a corrupt store or a password mismatch can cause Cassandra to fail
> accepting new connections as we throw away the old {{SslContext}}. This patch
> will ensure that we check the sanity of the certificates during startup and
> during hot reloading. This should protect against bad key/trust stores. As
> part of this PR, I have cleaned up the code a bit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14991) SSL Cert Hot Reloading should check for sanity of the new keystore/truststore before loading it
[
https://issues.apache.org/jira/browse/CASSANDRA-14991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746663#comment-16746663
]
Dinesh Joshi commented on CASSANDRA-14991:
--
dtest with vnodes have some failures but they're unrelated to this change.
dtests without vnodes and utests are good.
> SSL Cert Hot Reloading should check for sanity of the new keystore/truststore
> before loading it
> ---
>
> Key: CASSANDRA-14991
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14991
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Encryption
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Major
> Labels: security
>
> SSL Cert Hot Reloading assumes that the keystore & truststore are valid.
> However, a corrupt store or a password mismatch can cause Cassandra to fail
> accepting new connections as we throw away the old {{SslContext}}. This patch
> will ensure that we check the sanity of the certificates during startup and
> during hot reloading. This should protect against bad key/trust stores. As
> part of this PR, I have cleaned up the code a bit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13508) Make system.paxos table compaction strategy configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746655#comment-16746655
]
Ariel Weisberg commented on CASSANDRA-13508:
I don't think a paxos table per table is the right way to go about it. The key
for the paxos table should just be the partition key. If multiple tables share
the same partition key then you can do transactions across them. We want to
head in the direction of more expressive not less.
> Make system.paxos table compaction strategy configurable
>
>
> Key: CASSANDRA-13508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13508
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Coordination
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Major
> Labels: LWT, core, paxos
> Fix For: 4.x
>
> Attachments: test11.png, test2.png
>
>
> The default compaction strategy for {{system.paxos}} table is LCS for
> performance reason: CASSANDRA-7753. But for CAS heavily used cluster, the
> system is busy with {{system.paxos}} compaction.
> As the data in {{paxos}} table are TTL'ed, TWCS might be a better fit. In our
> test, it significantly reduced the number of compaction without impacting the
> latency too much:
> !test11.png!
> The time window for TWCS is set to 2 minutes for the test.
> Here is the p99 latency impact:
> !test2.png!
> the yellow one is LCS, the purple one is TWCS. Average p99 has about 10%
> increase.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14988) Building javadoc with Java11 fails
[
https://issues.apache.org/jira/browse/CASSANDRA-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremy Hanna updated CASSANDRA-14988:
-
Labels: Java11 (was: )
> Building javadoc with Java11 fails
> --
>
> Key: CASSANDRA-14988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14988
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation/Javadoc
>Reporter: Tommy Stendahl
>Assignee: Tommy Stendahl
>Priority: Blocker
> Labels: Java11
> Fix For: 4.0
>
>
> When building trunk with Java11 building javadoc fails with this error:
> {noformat}
> [javadoc]
> /repos/tmp/cassandra/src/java/org/apache/cassandra/hints/HintsBufferPool.java:28:
> error: package sun.nio.ch is not visible
> [javadoc] import sun.nio.ch.DirectBuffer;
> [javadoc] ^
> [javadoc] (package sun.nio.ch is declared in module java.base, which does not
> export it to the unnamed module)
> [javadoc] 1 error{noformat}
> This import is unused and was probably added by mistake, removing it fixes
> the problem.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14991) SSL Cert Hot Reloading should check for sanity of the new keystore/truststore before loading it
[
https://issues.apache.org/jira/browse/CASSANDRA-14991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-14991:
---
Reviewers: Ariel Weisberg
Reviewer: Ariel Weisberg
> SSL Cert Hot Reloading should check for sanity of the new keystore/truststore
> before loading it
> ---
>
> Key: CASSANDRA-14991
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14991
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Encryption
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Major
> Labels: security
>
> SSL Cert Hot Reloading assumes that the keystore & truststore are valid.
> However, a corrupt store or a password mismatch can cause Cassandra to fail
> accepting new connections as we throw away the old {{SslContext}}. This patch
> will ensure that we check the sanity of the certificates during startup and
> during hot reloading. This should protect against bad key/trust stores. As
> part of this PR, I have cleaned up the code a bit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14991) SSL Cert Hot Reloading should check for sanity of the new keystore/truststore before loading it
[
https://issues.apache.org/jira/browse/CASSANDRA-14991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-14991:
-
Description: SSL Cert Hot Reloading assumes that the keystore & truststore
are valid. However, a corrupt store or a password mismatch can cause Cassandra
to fail accepting new connections as we throw away the old {{SslContext}}. This
patch will ensure that we check the sanity of the certificates during startup
and during hot reloading. This should protect against bad key/trust stores. As
part of this PR, I have cleaned up the code a bit. (was: SSL Cert Hot
Reloading assumes that the keystore & truststore are valid. However, a corrupt
store or a password mismatch can cause Cassandra to fail accepting new
connections as we throw away the old {{SslContext}}. This patch will ensure
that we check the sanity of the certificates during startup and during hot
reloading. This should protect against bad key/trust stores.)
> SSL Cert Hot Reloading should check for sanity of the new keystore/truststore
> before loading it
> ---
>
> Key: CASSANDRA-14991
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14991
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Encryption
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Major
> Labels: security
>
> SSL Cert Hot Reloading assumes that the keystore & truststore are valid.
> However, a corrupt store or a password mismatch can cause Cassandra to fail
> accepting new connections as we throw away the old {{SslContext}}. This patch
> will ensure that we check the sanity of the certificates during startup and
> during hot reloading. This should protect against bad key/trust stores. As
> part of this PR, I have cleaned up the code a bit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14991) SSL Cert Hot Reloading should check for sanity of the new keystore/truststore before loading it
[
https://issues.apache.org/jira/browse/CASSANDRA-14991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746583#comment-16746583
]
Dinesh Joshi commented on CASSANDRA-14991:
--
||trunk||
|[branch|https://github.com/dineshjoshi/cassandra/tree/14991-trunk]|
|[utests
dtests|https://circleci.com/gh/dineshjoshi/workflows/cassandra/tree/14991-trunk]|
||
> SSL Cert Hot Reloading should check for sanity of the new keystore/truststore
> before loading it
> ---
>
> Key: CASSANDRA-14991
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14991
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Encryption
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Major
> Labels: security
>
> SSL Cert Hot Reloading assumes that the keystore & truststore are valid.
> However, a corrupt store or a password mismatch can cause Cassandra to fail
> accepting new connections as we throw away the old {{SslContext}}. This patch
> will ensure that we check the sanity of the certificates during startup and
> during hot reloading. This should protect against bad key/trust stores.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14991) SSL Cert Hot Reloading should check for sanity of the new keystore/truststore before loading it
Dinesh Joshi created CASSANDRA-14991:
Summary: SSL Cert Hot Reloading should check for sanity of the new
keystore/truststore before loading it
Key: CASSANDRA-14991
URL: https://issues.apache.org/jira/browse/CASSANDRA-14991
Project: Cassandra
Issue Type: Bug
Components: Feature/Encryption
Reporter: Dinesh Joshi
Assignee: Dinesh Joshi
SSL Cert Hot Reloading assumes that the keystore & truststore are valid.
However, a corrupt store or a password mismatch can cause Cassandra to fail
accepting new connections as we throw away the old {{SslContext}}. This patch
will ensure that we check the sanity of the certificates during startup and
during hot reloading. This should protect against bad key/trust stores.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14990) Move cqlsh tests to dtest repo
Dinesh Joshi created CASSANDRA-14990: Summary: Move cqlsh tests to dtest repo Key: CASSANDRA-14990 URL: https://issues.apache.org/jira/browse/CASSANDRA-14990 Project: Cassandra Issue Type: Bug Components: Test/dtest, Tool/cqlsh Reporter: Dinesh Joshi Assignee: Dinesh Joshi The cqlsh tests are split into the dtest repo and the main Cassandra repo. We should move them to dtests repo and migrate them from nosetests to pytests. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14985) CircleCI docker image should bake in more dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14985: --- Fix Version/s: (was: 3.11.x) (was: 3.0.x) (was: 2.1.x) 3.11.4 3.0.18 2.2.14 > CircleCI docker image should bake in more dependencies > > > Key: CASSANDRA-14985 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14985 > Project: Cassandra > Issue Type: Test > Components: Test/dtest, Test/unit >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Major > Fix For: 2.2.14, 3.0.18, 3.11.4, 4.0 > > > It's pretty frequent especially in the upgrade tests for either maven or > github to fail. I think they are detecting the thundering herd of containers > all pulling stuff at the same time and opting out. > We can reduce this especially for maven dependencies since most are hardly > ever changing by downloading everything we can in advance and baking it into > the image. > When building the docker image Initialize the local maven repository by > running ant maven-ant-tasks-retrieve-build for 2.1-trunk and do the same with > CCM and the Apache repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14985) CircleCI docker image should bake in more dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14985: --- Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as [cassandra 1d06b6686ff5a83f08b30eaf1968a3090f95df51|https://github.com/apache/cassandra/commit/1d06b6686ff5a83f08b30eaf1968a3090f95df51] and [cassandra-builds 82f14ad5dc01d3cebd0b20dee96ab4ab61cb8e86|https://github.com/apache/cassandra-builds/commit/82f14ad5dc01d3cebd0b20dee96ab4ab61cb8e86]. Thanks! > CircleCI docker image should bake in more dependencies > > > Key: CASSANDRA-14985 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14985 > Project: Cassandra > Issue Type: Test > Components: Test/dtest, Test/unit >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Major > Fix For: 4.0, 2.1.x, 3.0.x, 3.11.x > > > It's pretty frequent especially in the upgrade tests for either maven or > github to fail. I think they are detecting the thundering herd of containers > all pulling stuff at the same time and opting out. > We can reduce this especially for maven dependencies since most are hardly > ever changing by downloading everything we can in advance and baking it into > the image. > When building the docker image Initialize the local maven repository by > running ant maven-ant-tasks-retrieve-build for 2.1-trunk and do the same with > CCM and the Apache repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository.
aweisberg pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit f61b926b940f272ced0f2f6b2db2a9f3fd8d3599
Merge: db3c852 1d06b66
Author: Ariel Weisberg
AuthorDate: Fri Jan 18 12:39:50 2019 -0500
Merge branch 'cassandra-2.2' into cassandra-3.0
.circleci/config.yml | 2 +-
CHANGES.txt | 1 +
2 files changed, 2 insertions(+), 1 deletion(-)
diff --cc CHANGES.txt
index eeb3068,99da8f6..812af0a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,38 -1,11 +1,39 @@@
-2.2.14
+3.0.18
+ * Add a script to make running the cqlsh tests in cassandra repo easier
(CASSANDRA-14951)
+ * If SizeEstimatesRecorder misses a 'onDropTable' notification, the
size_estimates table will never be cleared for that table. (CASSANDRA-14905)
+ * Counters fail to increment in 2.1/2.2 to 3.X mixed version clusters
(CASSANDRA-14958)
+ * Streaming needs to synchronise access to LifecycleTransaction
(CASSANDRA-14554)
+ * Fix cassandra-stress write hang with default options (CASSANDRA-14616)
+ * Differentiate between slices and RTs when decoding legacy bounds
(CASSANDRA-14919)
+ * CommitLogReplayer.handleReplayError should print stack traces
(CASSANDRA-14589)
+ * Netty epoll IOExceptions caused by unclean client disconnects being logged
at INFO (CASSANDRA-14909)
+ * Unfiltered.isEmpty conflicts with Row extends AbstractCollection.isEmpty
(CASSANDRA-14588)
+ * RangeTombstoneList doesn't properly clean up mergeable or superseded rts
in some cases (CASSANDRA-14894)
+ * Fix handling of collection tombstones for dropped columns from legacy
sstables (CASSANDRA-14912)
+ * Throw exception if Columns serialized subset encode more columns than
possible (CASSANDRA-14591)
+ * Drop/add column name with different Kind can result in corruption
(CASSANDRA-14843)
+ * Fix missing rows when reading 2.1 SSTables with static columns in 3.0
(CASSANDRA-14873)
+ * Move TWCS message 'No compaction necessary for bucket size' to Trace level
(CASSANDRA-14884)
+ * Sstable min/max metadata can cause data loss (CASSANDRA-14861)
+ * Dropped columns can cause reverse sstable iteration to return prematurely
(CASSANDRA-14838)
+ * Legacy sstables with multi block range tombstones create invalid bound
sequences (CASSANDRA-14823)
+ * Expand range tombstone validation checks to multiple interim request
stages (CASSANDRA-14824)
+ * Reverse order reads can return incomplete results (CASSANDRA-14803)
+ * Avoid calling iter.next() in a loop when notifying indexers about range
tombstones (CASSANDRA-14794)
+ * Fix purging semi-expired RT boundaries in reversed iterators
(CASSANDRA-14672)
+ * DESC order reads can fail to return the last Unfiltered in the partition
(CASSANDRA-14766)
+ * Fix corrupted collection deletions for dropped columns in 3.0 <-> 2.{1,2}
messages (CASSANDRA-14568)
+ * Fix corrupted static collection deletions in 3.0 <-> 2.{1,2} messages
(CASSANDRA-14568)
+ * Handle failures in parallelAllSSTableOperation
(cleanup/upgradesstables/etc) (CASSANDRA-14657)
+ * Improve TokenMetaData cache populating performance avoid long locking
(CASSANDRA-14660)
+ * Backport: Flush netty client messages immediately (not by default)
(CASSANDRA-13651)
+ * Fix static column order for SELECT * wildcard queries (CASSANDRA-14638)
+ * sstableloader should use discovered broadcast address to connect
intra-cluster (CASSANDRA-14522)
+ * Fix reading columns with non-UTF names from schema (CASSANDRA-14468)
+ Merged from 2.2:
+ * CircleCI docker image should bake in more dependencies (CASSANDRA-14985)
* Don't enable client transports when bootstrap is pending (CASSANDRA-14525)
* MigrationManager attempts to pull schema from different major version
nodes (CASSANDRA-14928)
- * Don't skip entire sstables when reading backwards with mixed clustering
column order
- (CASSANDRA-14910)
- * Cannot perform slice reads in reverse direction against tables with
clustering columns
- in mixed order (CASSANDRA-14899)
* Fix incorrect cqlsh results when selecting same columns multiple times
(CASSANDRA-13262)
* Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377)
Merged from 2.1:
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 609af8c2e5ebb46a1c7120bf5201ef4c2f511e38 Merge: b875399 b51d3c9 Author: Ariel Weisberg AuthorDate: Fri Jan 18 12:41:01 2019 -0500 Merge branch 'cassandra-3.11' into trunk .circleci/config.yml | 2 +- CHANGES.txt | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --cc CHANGES.txt index 8e0b0c7,d85a4af..706a3ed --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -373,7 -33,10 +373,8 @@@ Merged from 3.0 * sstableloader should use discovered broadcast address to connect intra-cluster (CASSANDRA-14522) * Fix reading columns with non-UTF names from schema (CASSANDRA-14468) Merged from 2.2: + * CircleCI docker image should bake in more dependencies (CASSANDRA-14985) - * Don't enable client transports when bootstrap is pending (CASSANDRA-14525) * MigrationManager attempts to pull schema from different major version nodes (CASSANDRA-14928) - * Fix incorrect cqlsh results when selecting same columns multiple times (CASSANDRA-13262) * Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377) Merged from 2.1: * Paged Range Slice queries with DISTINCT can drop rows from results (CASSANDRA-14956) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (db3c852 -> f61b926)
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from db3c852 Merge branch 'cassandra-2.2' into cassandra-3.0 new 1d06b66 CircleCI docker image should bake in more dependencies new f61b926 Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .circleci/config.yml | 2 +- CHANGES.txt | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (e5ab9b6 -> b51d3c9)
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from e5ab9b6 Merge branch 'cassandra-3.0' into cassandra-3.11 new 1d06b66 CircleCI docker image should bake in more dependencies new f61b926 Merge branch 'cassandra-2.2' into cassandra-3.0 new b51d3c9 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .circleci/config.yml | 2 +- CHANGES.txt | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (b875399 -> 609af8c)
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from b875399 Merge branch 'cassandra-3.11' into trunk new 1d06b66 CircleCI docker image should bake in more dependencies new f61b926 Merge branch 'cassandra-2.2' into cassandra-3.0 new b51d3c9 Merge branch 'cassandra-3.0' into cassandra-3.11 new 609af8c Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .circleci/config.yml | 2 +- CHANGES.txt | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: CircleCI docker image should bake in more dependencies
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a commit to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-2.2 by this push: new 1d06b66 CircleCI docker image should bake in more dependencies 1d06b66 is described below commit 1d06b6686ff5a83f08b30eaf1968a3090f95df51 Author: Ariel Weisberg AuthorDate: Thu Jan 17 14:23:11 2019 -0500 CircleCI docker image should bake in more dependencies Patch by Ariel Weisberg; Reviewed by Stefan Podkowinski for CASSANDRA-14985 --- .circleci/config.yml | 2 +- CHANGES.txt | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 78dae4c..44707e2 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -75,7 +75,7 @@ workflows: #build_and_run_tests: *with_dtest_jobs_only #build_and_run_tests: *with_dtest_jobs #build_and_run_tests: *upgrade_job_only -docker_image: _image spod/cassandra-testing-ubuntu18-java11 +docker_image: _image aweisberg/cassandra-testing-ubuntu18-java11-w-dependencies version: 2 jobs: build: diff --git a/CHANGES.txt b/CHANGES.txt index 0260c75..99da8f6 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.14 + * CircleCI docker image should bake in more dependencies (CASSANDRA-14985) * Don't enable client transports when bootstrap is pending (CASSANDRA-14525) * MigrationManager attempts to pull schema from different major version nodes (CASSANDRA-14928) * Don't skip entire sstables when reading backwards with mixed clustering column order - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit b51d3c985ea7819e9e81e10ebeb36fc43142c245 Merge: e5ab9b6 f61b926 Author: Ariel Weisberg AuthorDate: Fri Jan 18 12:40:05 2019 -0500 Merge branch 'cassandra-3.0' into cassandra-3.11 .circleci/config.yml | 2 +- CHANGES.txt | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14985) CircleCI docker image should bake in more dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14985: --- Fix Version/s: 3.11.x 3.0.x 2.1.x > CircleCI docker image should bake in more dependencies > > > Key: CASSANDRA-14985 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14985 > Project: Cassandra > Issue Type: Test > Components: Test/dtest, Test/unit >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Major > Fix For: 4.0, 2.1.x, 3.0.x, 3.11.x > > > It's pretty frequent especially in the upgrade tests for either maven or > github to fail. I think they are detecting the thundering herd of containers > all pulling stuff at the same time and opting out. > We can reduce this especially for maven dependencies since most are hardly > ever changing by downloading everything we can in advance and baking it into > the image. > When building the docker image Initialize the local maven repository by > running ant maven-ant-tasks-retrieve-build for 2.1-trunk and do the same with > CCM and the Apache repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master updated: CircleCI docker image should bake in more dependencies
This is an automated email from the ASF dual-hosted git repository.
aweisberg pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/master by this push:
new 82f14ad CircleCI docker image should bake in more dependencies
82f14ad is described below
commit 82f14ad5dc01d3cebd0b20dee96ab4ab61cb8e86
Author: Ariel Weisberg
AuthorDate: Thu Jan 17 14:22:18 2019 -0500
CircleCI docker image should bake in more dependencies
Patch by Ariel Weisberg; Reviewed by Stefan Podkowinski for CASSANDRA-14985
---
docker/testing/ubuntu18_j11.docker| 2 +-
docker/testing/ubuntu18_j11_w_dependencies.docker | 37 +++
2 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/docker/testing/ubuntu18_j11.docker
b/docker/testing/ubuntu18_j11.docker
index 30ecdf1..d54eb5e 100644
--- a/docker/testing/ubuntu18_j11.docker
+++ b/docker/testing/ubuntu18_j11.docker
@@ -19,7 +19,7 @@ MAINTAINER Stefan Podkowinski
RUN export DEBIAN_FRONTEND=noninteractive && \
apt-get update && \
-apt-get install -y --no-install-recommends software-properties-common
apt-utils
+apt-get install -y --no-install-recommends software-properties-common
apt-utils vim
RUN export DEBIAN_FRONTEND=noninteractive && \
apt-get update && \
diff --git a/docker/testing/ubuntu18_j11_w_dependencies.docker
b/docker/testing/ubuntu18_j11_w_dependencies.docker
new file mode 100644
index 000..0aaead9
--- /dev/null
+++ b/docker/testing/ubuntu18_j11_w_dependencies.docker
@@ -0,0 +1,37 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# base things off the testing image without dependencies warmed up
+FROM aweisberg/cassandra-testing-ubuntu18-java11
+MAINTAINER Stefan Podkowinski
+
+USER cassandra
+ENV HOME /home/cassandra
+WORKDIR /home/cassandra
+
+# Fetch the maven dependencies in advance since this tends to fail at runtime
+ARG CASSANDRA_GIT_URL=https://gitbox.apache.org/repos/asf/cassandra.git
+RUN /bin/bash -c "git clone ${CASSANDRA_GIT_URL} ~/cassandra && \
+cd ~/cassandra && ant maven-ant-tasks-retrieve-build && \
+git checkout origin/cassandra-3.11 && ant maven-ant-tasks-retrieve-build
&& \
+git checkout origin/cassandra-3.0 && ant maven-ant-tasks-retrieve-build &&
\
+git checkout origin/cassandra-2.2 && ant maven-ant-tasks-retrieve-build &&
\
+git checkout origin/cassandra-2.1 && ant maven-ant-tasks-retrieve-build &&
\
+rm -fr ~/cassandra"
+
+# Initialize the CCM git repo as well as this also can fail to clone
+RUN /bin/bash -c "source ~/env/bin/activate && \
+ccm create -n 1 -v git:trunk test && ccm remove test && \
+ccm create -n 1 -v git:cassandra-3.11 test && ccm remove test && \
+ccm create -n 1 -v git:cassandra-3.0 test && ccm remove test && \
+ccm create -n 1 -v git:cassandra-2.2 test && ccm remove test && \
+ccm create -n 1 -v git:cassandra-2.1 test && ccm remove test"
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14937) Multi-version In-JVM dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-14937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746451#comment-16746451
]
Benedict edited comment on CASSANDRA-14937 at 1/18/19 4:27 PM:
---
Thanks for the quick review! Before I had even had a chance to post a comment
explaining the latest changes.
These are mostly excellent points, but a few quibbles:
{quote}it seems that IMessage doesn't have to be interface with each version
having it's own implementation
{quote}
This is necessary to support cross-version compatibility. There needs to be a
common parent that is shared across the class loaders, so that older versions
can access the contents of the newer messages (and vice versa)
{quote}new Object[][] here can be constant
{quote}
I deliberately chose not to do that here, since this is an uncommon code path,
it's test code, and it's unnecessary boilerplate. I don't mind changing if you
have a strong opinion on it though.
{quote}probably this will be documented, but "build" directory is currently
assumed to be under resources or?
{quote}
The build directory is in the project root, i.e. where {{ant dtest-jar}} writes
it. Probably this needs a wiki page or something documenting it, besides some
improved javadoc (that is in progress)
{quote}if you at 3.0 branch, you can't start 3.11 or 4.0 cluster because of
CDC. On the more general note. Since configs are generated from "cluster"
perspective (in AbstractCluster#create, we do not get to pick a right config
for the version.
{quote}
It's only intended to be backwards compatible, not forwards. Possibly you were
mislead by the UpgradeTest which may have been backported late last night after
extensive refactoring to the patch yesterday - I haven't yet cleaned up the
final touches of the various ports.
There's also backwards compatibility translations baked into the
InstanceConfig; for most fields this is managed by simply ignoring missing
properties when populating the config, and some specific legacy properties
being specified in all future versions as well. For 4.0 there is also special
version-aware logic to strip the seed provider list of its ports.
{quote}InstanceClassLoader#id is now unused
{quote}
As discussed on a previous JIRA, the idea of this was to aid in debugging. But
now we annotate the threads with the information, it's probably unnecessary, so
will remove it.
{quote}it looks like right now, to test current version
{quote}
No, you can already use the Versions.CURRENT public static final field, as is
used for the regular DistributedReadWritePathTest, but it's a TODO to
automatically include that in the Versions.find() (which currently only looks
for jars). Thanks for reminding me.
{quote}it looks like we can not create mixed-version cluster at the moment even
though nothing prevents it. Maybe it was planned for future? For example,
create a cluster with 3.0 and 4.0 without upgrades. I see quite a few use-cases
for that.
{quote}
I originally had planned to support that, but rolled back the decision since it
logically didn't seem to make much sense. We definitely don't support starting
a brand new cluster with a mixed version? We can reintroduce it if we want, but
it's fairly straight forward to stop/start the node with a new version too.
I'll address the other items you point out shortly.
was (Author: benedict):
Thanks for the quick review! Before I had even had a chance to post a comment
explaining the latest changes.
These are mostly excellent points, but a few quibbles:
{quote}it seems that IMessage doesn't have to be interface with each version
having it's own implementation
{quote}
This is necessary to support cross-version compatibility. There needs to be a
common parent that is shared across the class loaders, so that older versions
can access the contents of the newer messages (and vice versa)
{quote}new Object[][] here can be constant
{quote}
I deliberately chose not to do that here, since this is an uncommon code path,
it's test code, and it's unnecessary boilerplate. I don't mind changing if you
have a strong opinion on it though.
{quote}probably this will be documented, but "build" directory is currently
assumed to be under resources or?
{quote}
The build directory is in the project root, i.e. where {{ant dtest-jar}} writes
it. Probably this needs a wiki page or something documenting it, besides some
improved javadoc (that is in progress)
{quote}if you at 3.0 branch, you can't start 3.11 or 4.0 cluster because of
CDC. On the more general note. Since configs are generated from "cluster"
perspective (in AbstractCluster#create, we do not get to pick a right config
for the version.
{quote}
It's only intended to be backwards compatible, not forwards. Possibly you were
mislead by the UpgradeTest which may have been backported late last night after
extensive refactoring to the patch yesterday - I
[jira] [Commented] (CASSANDRA-14937) Multi-version In-JVM dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-14937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746451#comment-16746451
]
Benedict commented on CASSANDRA-14937:
--
Thanks for the quick review! Before I had even had a chance to post a comment
explaining the latest changes.
These are mostly excellent points, but a few quibbles:
{quote}it seems that IMessage doesn't have to be interface with each version
having it's own implementation
{quote}
This is necessary to support cross-version compatibility. There needs to be a
common parent that is shared across the class loaders, so that older versions
can access the contents of the newer messages (and vice versa)
{quote}new Object[][] here can be constant
{quote}
I deliberately chose not to do that here, since this is an uncommon code path,
it's test code, and it's unnecessary boilerplate. I don't mind changing if you
have a strong opinion on it though.
{quote}probably this will be documented, but "build" directory is currently
assumed to be under resources or?
{quote}
The build directory is in the project root, i.e. where {{ant dtest-jar}} writes
it. Probably this needs a wiki page or something documenting it, besides some
improved javadoc (that is in progress)
{quote}if you at 3.0 branch, you can't start 3.11 or 4.0 cluster because of
CDC. On the more general note. Since configs are generated from "cluster"
perspective (in AbstractCluster#create, we do not get to pick a right config
for the version.
{quote}
It's only intended to be backwards compatible, not forwards. Possibly you were
mislead by the UpgradeTest which may have been backported late last night after
extensive refactoring to the patch yesterday - I haven't yet cleaned up the
final touches of the various ports.
{quote}InstanceClassLoader#id is now unused
{quote}
As discussed on a previous JIRA, the idea of this was to aid in debugging. But
now we annotate the threads with the information, it's probably unnecessary, so
will remove it.
{quote}it looks like right now, to test current version
{quote}
No, you can already use the Versions.CURRENT public static final field, as is
used for the regular DistributedReadWritePathTest, but it's a TODO to
automatically include that in the Versions.find() (which currently only looks
for jars). Thanks for reminding me.
{quote}it looks like we can not create mixed-version cluster at the moment even
though nothing prevents it. Maybe it was planned for future? For example,
create a cluster with 3.0 and 4.0 without upgrades. I see quite a few use-cases
for that.
{quote}
I originally had planned to support that, but rolled back the decision since it
logically didn't seem to make much sense. We definitely don't support starting
a brand new cluster with a mixed version? We can reintroduce it if we want, but
it's fairly straight forward to stop/start the node with a new version too.
I'll address the other items you point out shortly.
> Multi-version In-JVM dtests
> ---
>
> Key: CASSANDRA-14937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14937
> Project: Cassandra
> Issue Type: New Feature
> Components: Test/dtest
>Reporter: Benedict
>Assignee: Benedict
>Priority: Major
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> In order to support more sophisticated upgrade tests, including complex fuzz
> tests that can span a sequence of version upgrades, we propose abstracting a
> cross-version API for the in-jvm dtests. This will permit starting a node
> with an arbitrary compatible C* version, stopping the node, and restarting it
> with another C* version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14937) Multi-version In-JVM dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-14937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746436#comment-16746436
]
Alex Petrov commented on CASSANDRA-14937:
-
Thank you for the patch, I like the direction and think it will be a great
addition!
* {{InvokableInstance}} serves no other purpose than add implementation to
InvokableInstance methods. We can use default implementations on interface and
get rid of this class alltogether.
*
[here|https://github.com/apache/cassandra/compare/trunk...belliottsmith:14937-3.0#diff-3a4f5b9b5507554db09adad8022b0811R35]
we do not need to have a full package name
{{org.apache.cassandra.distributed.api.}}
* it seems that {{IMessage}} doesn't have to be interface with each version
having it's own implementation
* in {{Listen}}, outer loop doesn't look like necessary
[here|https://github.com/apache/cassandra/compare/trunk...belliottsmith:14937-3.0#diff-8b99167f6379cc470fcbaf5f640a098eR38],
also package name can be removed
[here|https://github.com/apache/cassandra/compare/trunk...belliottsmith:14937-3.0#diff-8b99167f6379cc470fcbaf5f640a098eR26]
* {{new Object[][]}}
[here|https://github.com/apache/cassandra/compare/trunk...belliottsmith:14937-3.0#diff-3a4f5b9b5507554db09adad8022b0811R70]
can be constant
* VERSION_4
[here|https://github.com/apache/cassandra/compare/trunk...belliottsmith:14937-3.0#diff-3a4f5b9b5507554db09adad8022b0811R62]
probably should be CURRENT_VERSION
* probably this will be documented, but "build" directory is currently
assumed to be under resources or?
* maybe for debugging purposes we could list found versions (also, we could
do it once per test class)
* if you at 3.0 branch, you can't start 3.11 or 4.0 cluster because of CDC.
On the more general note. Since configs are generated from "cluster"
perspective (in {{AbstractCluster#create}}, we do not get to pick a right
config for the version.
* Exceptions on when instance can't be load are currently not very verbose,
you just get NPE. It'd be great if we could say "couldn't find your version,
please put it there and there".
* trunk produces extra blank line after each log line and kind of has a weird
formatting, looks like something might be double-wrapped:
{code}
INFO [node3_isolatedExecutor:2] 2019-01-18 16:49:19,848
SubstituteLogger.java:169 - DEBUG [node3_isolatedExecutor:2] node3 2019-01-18
16:49:19,845 DatabaseDescriptor.java:307 - Syncing log with a batch window of
1.0
{code}
* {{InstanceClassLoader#id}} is now unused
* {{InstanceClassLoader#sharedPackageNames}} is a predicate, but is named as
noun, it'd be great to change the name.
* it looks like right now, to test _current_ version, you have to re-package.
But this shouldn't be the case, we can just use current version class path
without wrapping it into jar. This could really improve development cycle,
especially when you know which version "misbehaves"
* it looks like we can not create mixed-version cluster at the moment even
though nothing prevents it. Maybe it was planned for future? For example,
create a cluster with 3.0 and 4.0 without upgrades. I see quite a few use-cases
for that.
> Multi-version In-JVM dtests
> ---
>
> Key: CASSANDRA-14937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14937
> Project: Cassandra
> Issue Type: New Feature
> Components: Test/dtest
>Reporter: Benedict
>Assignee: Benedict
>Priority: Major
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> In order to support more sophisticated upgrade tests, including complex fuzz
> tests that can span a sequence of version upgrades, we propose abstracting a
> cross-version API for the in-jvm dtests. This will permit starting a node
> with an arbitrary compatible C* version, stopping the node, and restarting it
> with another C* version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14985) CircleCI docker image should bake in more dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Podkowinski updated CASSANDRA-14985: --- Status: Ready to Commit (was: Patch Available) > CircleCI docker image should bake in more dependencies > > > Key: CASSANDRA-14985 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14985 > Project: Cassandra > Issue Type: Test > Components: Test/dtest, Test/unit >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Major > Fix For: 4.0 > > > It's pretty frequent especially in the upgrade tests for either maven or > github to fail. I think they are detecting the thundering herd of containers > all pulling stuff at the same time and opting out. > We can reduce this especially for maven dependencies since most are hardly > ever changing by downloading everything we can in advance and baking it into > the image. > When building the docker image Initialize the local maven repository by > running ant maven-ant-tasks-retrieve-build for 2.1-trunk and do the same with > CCM and the Apache repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14985) CircleCI docker image should bake in more dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746407#comment-16746407 ] Stefan Podkowinski commented on CASSANDRA-14985: Just to be clear, I'm not at all against updating the base image or caching dependencies. But I'd rather avoid the situation where someone only wants to update cached dependencies and accidentally updates other packages from the main image as well. CCM is a good example, as updating CCM was not a goal of this ticket and yet still happened, just by touching the base image. So lets just have two ways of either explicitly updating the base or caching image and just avoid these issues. Anyways, feel free to commit the added separate Dockerfile. > CircleCI docker image should bake in more dependencies > > > Key: CASSANDRA-14985 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14985 > Project: Cassandra > Issue Type: Test > Components: Test/dtest, Test/unit >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Major > Fix For: 4.0 > > > It's pretty frequent especially in the upgrade tests for either maven or > github to fail. I think they are detecting the thundering herd of containers > all pulling stuff at the same time and opting out. > We can reduce this especially for maven dependencies since most are hardly > ever changing by downloading everything we can in advance and baking it into > the image. > When building the docker image Initialize the local maven repository by > running ant maven-ant-tasks-retrieve-build for 2.1-trunk and do the same with > CCM and the Apache repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14931) Backport In-JVM dtests to 2.2, 3.0 and 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-14931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746243#comment-16746243
]
Alex Petrov commented on CASSANDRA-14931:
-
Thank you for the patch!
Since this was largely reviewed and these are just backports, it was relatively
simple to review those.
+1 with several minor things:
* 2.2 branch tests are failing with the same exception wrapping failure we've
discussed in [CASSANDRA-14922] followup.
* Older branches are slightly different from trunk in terms of MBean
registration: in 4.0 {{PermissionsCache}} and {{RolesCache}} inherit from
{{AuthCache}}; so we need to use {{MBeanWrapper}} in these two classes in older
version. We do not have tests that use those just yet so we didn't have
automated means to detect it, but it's also simple enough to fix.
* You remove
[static|https://github.com/belliottsmith/cassandra/commit/4748a6d61b9715de87cd5cc20acc9f09391b4348#diff-d4e3b82e9bebfd2cb466b4a30af07fa4L118]
modified here but it is present in 4.0. I'm ok either way, just making sure it
was intentional (and probably it makes sense to make it uniform).
* On 3.11 it looks like logging initialisation has changed / is different
from the rest of branches. (I assume) Per classloader we get the following:
{code}
SLF4J: The following set of substitute loggers may have been accessed
SLF4J: during the initialization phase. Logging calls during this
SLF4J: phase were not honored. However, subsequent logging calls to these
SLF4J: loggers will work as normally expected.
SLF4J: See also http://www.slf4j.org/codes.html#substituteLogger
{code}
> Backport In-JVM dtests to 2.2, 3.0 and 3.11
> ---
>
> Key: CASSANDRA-14931
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14931
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest
>Reporter: Benedict
>Assignee: Benedict
>Priority: Major
> Fix For: 2.2.14, 3.0.18, 3.11.4
>
>
> The In-JVM dtests are of significant value, and much of the testing we are
> exploring with it can easily be utilised on all presently maintained
> versions. We should backport the functionality to at least 3.0.x and 3.11.x
> - and perhaps even consider 2.2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
