[jira] [Comment Edited] (CASSANDRA-18773) Compactions are slow
[
https://issues.apache.org/jira/browse/CASSANDRA-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755804#comment-17755804
]
Cameron Zemek edited comment on CASSANDRA-18773 at 8/18/23 5:05 AM:
!flamegraph.png|width=1508,height=691!
was (Author: cam1982):
!flamegraph.png!
> Compactions are slow
>
>
> Key: CASSANDRA-18773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18773
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction
>Reporter: Cameron Zemek
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Attachments: flamegraph.png, stress.yaml
>
>
> I have noticed that compactions involving a lot of sstables are very slow
> (for example major compactions). I have attached a cassandra stress profile
> that can generate such a dataset under ccm. In my local test I have 2567
> sstables at 4Mb each.
> I added code to track wall clock time of various parts of the code. One
> problematic part is ManyToOne constructor. Tracing through the code for every
> partition creating a ManyToOne for all the sstable iterators for each
> partition. In my local test get a measy 60Kb/sec read speed, and bottlenecked
> on single core CPU (since this code is single threaded) with it spending 85%
> of the wall clock time in ManyToOne constructor.
> As another datapoint to show its the merge iterator part of the code using
> the cfstats from [https://github.com/instaclustr/cassandra-sstable-tools/]
> which reads all the sstables but does no merging gets 26Mb/sec read speed.
> Tracking back from ManyToOne call I see this in
> UnfilteredPartitionIterators::merge
> {code:java}
> for (int i = 0; i < toMerge.size(); i++)
> {
> if (toMerge.get(i) == null)
> {
> if (null == empty)
> empty = EmptyIterators.unfilteredRow(metadata,
> partitionKey, isReverseOrder);
> toMerge.set(i, empty);
> }
> }
> {code}
> Not sure what purpose of creating these empty rows are. But on a whim I
> removed all these empty iterators before passing to ManyToOne and then all
> the wall clock time shifted to CompactionIterator::hasNext() and read speed
> increased to 1.5Mb/s.
> So there are further bottlenecks in this code path it seems, but the first is
> this ManyToOne and having to build it for every partition read.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18773) Compactions are slow
[
https://issues.apache.org/jira/browse/CASSANDRA-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cameron Zemek updated CASSANDRA-18773:
--
Attachment: flamegraph.png
> Compactions are slow
>
>
> Key: CASSANDRA-18773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18773
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction
>Reporter: Cameron Zemek
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Attachments: flamegraph.png, stress.yaml
>
>
> I have noticed that compactions involving a lot of sstables are very slow
> (for example major compactions). I have attached a cassandra stress profile
> that can generate such a dataset under ccm. In my local test I have 2567
> sstables at 4Mb each.
> I added code to track wall clock time of various parts of the code. One
> problematic part is ManyToOne constructor. Tracing through the code for every
> partition creating a ManyToOne for all the sstable iterators for each
> partition. In my local test get a measy 60Kb/sec read speed, and bottlenecked
> on single core CPU (since this code is single threaded) with it spending 85%
> of the wall clock time in ManyToOne constructor.
> As another datapoint to show its the merge iterator part of the code using
> the cfstats from [https://github.com/instaclustr/cassandra-sstable-tools/]
> which reads all the sstables but does no merging gets 26Mb/sec read speed.
> Tracking back from ManyToOne call I see this in
> UnfilteredPartitionIterators::merge
> {code:java}
> for (int i = 0; i < toMerge.size(); i++)
> {
> if (toMerge.get(i) == null)
> {
> if (null == empty)
> empty = EmptyIterators.unfilteredRow(metadata,
> partitionKey, isReverseOrder);
> toMerge.set(i, empty);
> }
> }
> {code}
> Not sure what purpose of creating these empty rows are. But on a whim I
> removed all these empty iterators before passing to ManyToOne and then all
> the wall clock time shifted to CompactionIterator::hasNext() and read speed
> increased to 1.5Mb/s.
> So there are further bottlenecks in this code path it seems, but the first is
> this ManyToOne and having to build it for every partition read.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18773) Compactions are slow
[
https://issues.apache.org/jira/browse/CASSANDRA-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755804#comment-17755804
]
Cameron Zemek commented on CASSANDRA-18773:
---
!flamegraph.png!
> Compactions are slow
>
>
> Key: CASSANDRA-18773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18773
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction
>Reporter: Cameron Zemek
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Attachments: flamegraph.png, stress.yaml
>
>
> I have noticed that compactions involving a lot of sstables are very slow
> (for example major compactions). I have attached a cassandra stress profile
> that can generate such a dataset under ccm. In my local test I have 2567
> sstables at 4Mb each.
> I added code to track wall clock time of various parts of the code. One
> problematic part is ManyToOne constructor. Tracing through the code for every
> partition creating a ManyToOne for all the sstable iterators for each
> partition. In my local test get a measy 60Kb/sec read speed, and bottlenecked
> on single core CPU (since this code is single threaded) with it spending 85%
> of the wall clock time in ManyToOne constructor.
> As another datapoint to show its the merge iterator part of the code using
> the cfstats from [https://github.com/instaclustr/cassandra-sstable-tools/]
> which reads all the sstables but does no merging gets 26Mb/sec read speed.
> Tracking back from ManyToOne call I see this in
> UnfilteredPartitionIterators::merge
> {code:java}
> for (int i = 0; i < toMerge.size(); i++)
> {
> if (toMerge.get(i) == null)
> {
> if (null == empty)
> empty = EmptyIterators.unfilteredRow(metadata,
> partitionKey, isReverseOrder);
> toMerge.set(i, empty);
> }
> }
> {code}
> Not sure what purpose of creating these empty rows are. But on a whim I
> removed all these empty iterators before passing to ManyToOne and then all
> the wall clock time shifted to CompactionIterator::hasNext() and read speed
> increased to 1.5Mb/s.
> So there are further bottlenecks in this code path it seems, but the first is
> this ManyToOne and having to build it for every partition read.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (0e7fe18e -> 7cd42fae)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 0e7fe18e generate docs for cceb2c06 new 7cd42fae generate docs for cceb2c06 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (0e7fe18e) \ N -- N -- N refs/heads/asf-staging (7cd42fae) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4852435 -> 4852435 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] (CASSANDRA-18778) Empty keystore_password no longer allowed on encryption_options
[ https://issues.apache.org/jira/browse/CASSANDRA-18778 ]
Stefan Miklosovic deleted comment on CASSANDRA-18778:
---
was (Author: smiklosovic):
_which should involve changing FileBasedSslContextFactory.validatePassword to
only disallow null passwords_
I dont understand, in 18124 we made them nullable?
> Empty keystore_password no longer allowed on encryption_options
> ---
>
> Key: CASSANDRA-18778
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18778
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Normal
> Fix For: 4.1.x, 5.0.x
>
>
> After CASSANDRA-18124 (introduced in 4.1.2 and 5.0) it is no longer possible
> to set an empty {{keystore_password}} under {{client_encryption_options}} or
> {{server_encryption_options}} using the default implementation
> {{{}DefaultSslContextFactory{}}}.
> While keytool does not allow generating keystores with empty passwords, it
> does support reading them. It is not uncommon to use PKCS12 certificates
> generated by other tools (eg. openssl) that do not enforce passwords.
> The fix for this should be pretty straightforward, which should involve
> changing
> [FileBasedSslContextFactory.validatePassword|https://github.com/apache/cassandra/blob/cassandra-4.1.2/src/java/org/apache/cassandra/security/FileBasedSslContextFactory.java#L128-L135]
> to only disallow null passwords (which would be consistent with previous
> versions). I will create pull requests against the relevant branches shortly.
> {noformat}
> Exception (org.apache.cassandra.exceptions.ConfigurationException)
> encountered during startup: Failed to initialize SSL
> org.apache.cassandra.exceptions.ConfigurationException: Failed to initialize
> SSL
> at
> org.apache.cassandra.config.DatabaseDescriptor.applySslContext(DatabaseDescriptor.java:1155)
> at
> org.apache.cassandra.config.DatabaseDescriptor.applyAll(DatabaseDescriptor.java:390)
> at
> org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:204)
> at
> org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:188)
> at
> org.apache.cassandra.service.CassandraDaemon.applyConfig(CassandraDaemon.java:804)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:747)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:875)
> Caused by: java.io.IOException: Failed to create SSL context using Native
> transport
> at
> org.apache.cassandra.security.SSLFactory.validateSslContext(SSLFactory.java:405)
> at
> org.apache.cassandra.config.DatabaseDescriptor.applySslContext(DatabaseDescriptor.java:1150)
> ... 6 more
> Caused by: java.lang.IllegalArgumentException: 'keystore_password' must be
> specified
> at
> org.apache.cassandra.security.FileBasedSslContextFactory.validatePassword(FileBasedSslContextFactory.java:133)
> at
> org.apache.cassandra.security.FileBasedSslContextFactory.buildKeyManagerFactory(FileBasedSslContextFactory.java:151)
> at
> org.apache.cassandra.security.AbstractSslContextFactory.createNettySslContext(AbstractSslContextFactory.java:181)
> at
> org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:168)
> at
> org.apache.cassandra.security.SSLFactory.validateSslContext(SSLFactory.java:355)
> ... 7 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18745) cqlsh should warn on server version mismatch
[ https://issues.apache.org/jira/browse/CASSANDRA-18745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18745: - Test and Documentation Plan: run CI Status: Patch Available (was: Open) > cqlsh should warn on server version mismatch > > > Key: CASSANDRA-18745 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18745 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > > When cqlsh is used against a different version of the server than the one > that it shipped with, it should emit a warning so that users aren't confused > when some things don't work correctly. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18778) Empty keystore_password no longer allowed on encryption_options
[
https://issues.apache.org/jira/browse/CASSANDRA-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755698#comment-17755698
]
Stefan Miklosovic commented on CASSANDRA-18778:
---
_which should involve changing FileBasedSslContextFactory.validatePassword to
only disallow null passwords_
I dont understand, in 18124 we made them nullable?
> Empty keystore_password no longer allowed on encryption_options
> ---
>
> Key: CASSANDRA-18778
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18778
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Normal
> Fix For: 4.1.x, 5.0.x
>
>
> After CASSANDRA-18124 (introduced in 4.1.2 and 5.0) it is no longer possible
> to set an empty {{keystore_password}} under {{client_encryption_options}} or
> {{server_encryption_options}} using the default implementation
> {{{}DefaultSslContextFactory{}}}.
> While keytool does not allow generating keystores with empty passwords, it
> does support reading them. It is not uncommon to use PKCS12 certificates
> generated by other tools (eg. openssl) that do not enforce passwords.
> The fix for this should be pretty straightforward, which should involve
> changing
> [FileBasedSslContextFactory.validatePassword|https://github.com/apache/cassandra/blob/cassandra-4.1.2/src/java/org/apache/cassandra/security/FileBasedSslContextFactory.java#L128-L135]
> to only disallow null passwords (which would be consistent with previous
> versions). I will create pull requests against the relevant branches shortly.
> {noformat}
> Exception (org.apache.cassandra.exceptions.ConfigurationException)
> encountered during startup: Failed to initialize SSL
> org.apache.cassandra.exceptions.ConfigurationException: Failed to initialize
> SSL
> at
> org.apache.cassandra.config.DatabaseDescriptor.applySslContext(DatabaseDescriptor.java:1155)
> at
> org.apache.cassandra.config.DatabaseDescriptor.applyAll(DatabaseDescriptor.java:390)
> at
> org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:204)
> at
> org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:188)
> at
> org.apache.cassandra.service.CassandraDaemon.applyConfig(CassandraDaemon.java:804)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:747)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:875)
> Caused by: java.io.IOException: Failed to create SSL context using Native
> transport
> at
> org.apache.cassandra.security.SSLFactory.validateSslContext(SSLFactory.java:405)
> at
> org.apache.cassandra.config.DatabaseDescriptor.applySslContext(DatabaseDescriptor.java:1150)
> ... 6 more
> Caused by: java.lang.IllegalArgumentException: 'keystore_password' must be
> specified
> at
> org.apache.cassandra.security.FileBasedSslContextFactory.validatePassword(FileBasedSslContextFactory.java:133)
> at
> org.apache.cassandra.security.FileBasedSslContextFactory.buildKeyManagerFactory(FileBasedSslContextFactory.java:151)
> at
> org.apache.cassandra.security.AbstractSslContextFactory.createNettySslContext(AbstractSslContextFactory.java:181)
> at
> org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:168)
> at
> org.apache.cassandra.security.SSLFactory.validateSslContext(SSLFactory.java:355)
> ... 7 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] (CASSANDRA-18736) Streaming exception race creates corrupt transaction log files that prevent restart
[ https://issues.apache.org/jira/browse/CASSANDRA-18736 ]
Caleb Rackliffe deleted comment on CASSANDRA-18736:
-
was (Author: maedhroz):
+1
> Streaming exception race creates corrupt transaction log files that prevent
> restart
> ---
>
> Key: CASSANDRA-18736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18736
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Local/Startup and Shutdown
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x
>
>
> On restart, Cassandra logs this message and terminates.
> {code:java}
> ERROR 2023-07-17T17:17:22,931 [main]
> org.apache.cassandra.db.lifecycle.LogTransaction:561 - Unexpected disk state:
> failed to read transaction log
> [nb_txn_stream_39d5f6b0-fb81-11ed-8f46-e97b3f61511e.log in
> /datadir1/keyspace/table-c9527530a0d611e8813f034699fc9043]
> Files and contents follow:
> /datadir1/keyspace/table-c9527530a0d611e8813f034699fc9043/nb_txn_stream_39d5f6b0-fb81-11ed-8f46-e97b3f61511e.log
> ABORT:[,0,0][737437348]
> ***This record should have been the last one in all replicas
>
> ADD:[/datadir1/keyspace/table-c9527530a0d611e8813f034699fc9043/nb-284490-big-,0,8][2493503833]
> {code}
> The root cause is a race during streaming exception handling.
> Although concurrent modification of to the {{LogTransaction}} was added for
> CASSANDRA-16225, there is nothing to prevent usage after the transaction is
> completed (committed/aborted) once it has been processed by
> {{TransactionTidier}} (after the last reference is released). Before the
> transaction is tidied, the {{LogFile}} keeps a list of records that are
> checked for completion before adding new entries. In {{TransactionTidier}}
> {{LogFile.records}} are cleared as no longer needed, however the
> LogTransaction/LogFile is still accessible to the stream.
> The changes in CASSANDRA-17273 added a parallel set of {{onDiskRecords}} that
> could be used to reliably recreate the transaction log at any new datadirs
> the same as the existing
> datadirs - regardless of the effect of
> {{LogTransaction.untrackNew/LogFile.remove}}
> If a streaming exception causes the LogTransaction to be aborted and tidied
> just before {{SimpleSSTableMultiWriter}} calls trackNew to add a new sstable.
> At the time of the call, the {{LogFile}} will not contain any {{LogReplicas}},
> {{LogFile.records}} will be empty, and {{LogFile.onDiskRecords}} will contain
> an {{ABORT}}.
> When {{LogTransaction.trackNew/LogFile.add}} is called, the check for
> completed transaction fails as records is empty, there are no replicas on the
> datadir, so {{maybeCreateReplicas}} creates a new txnlog file replica
> containing ABORT, then
> appends an ADD record.
> The LogFile has already been tidied after the abort so the txnlog file is not
> removed and sits on disk until a restart, causing the faiulre.
> There is a related exception caused with a different interleaving of aborts,
> after an sstable is added, however this is just a nuisance in the logs as the
> LogRelica is already created with an {{ADD}} record first.
> {code:java}
> java.lang.AssertionError:
> [ADD:[/datadir1/keyspace/table/nb-23314378-big-,0,8][1869379820]] is not
> tracked by 55be35b0-35d1-11ee-865d-8b1e3c48ca06
> at org.apache.cassandra.db.lifecycle.LogFile.remove(LogFile.java:388)
> at
> org.apache.cassandra.db.lifecycle.LogTransaction.untrackNew(LogTransaction.java:158)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.untrackNew(LifecycleTransaction.java:577)
> at
> org.apache.cassandra.db.streaming.CassandraStreamReceiver$1.untrackNew(CassandraStreamReceiver.java:149)
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.abort(SimpleSSTableMultiWriter.java:95)
> at
> org.apache.cassandra.io.sstable.format.RangeAwareSSTableWriter.abort(RangeAwareSSTableWriter.java:191)
> at
> org.apache.cassandra.db.streaming.CassandraCompressedStreamReader.read(CassandraCompressedStreamReader.java:115)
> at
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:85)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:53)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:38)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:53)
> at
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:172)
> at
>
[jira] [Commented] (CASSANDRA-18736) Streaming exception race creates corrupt transaction log files that prevent restart
[
https://issues.apache.org/jira/browse/CASSANDRA-18736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755697#comment-17755697
]
Caleb Rackliffe commented on CASSANDRA-18736:
-
+1
> Streaming exception race creates corrupt transaction log files that prevent
> restart
> ---
>
> Key: CASSANDRA-18736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18736
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Local/Startup and Shutdown
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x
>
>
> On restart, Cassandra logs this message and terminates.
> {code:java}
> ERROR 2023-07-17T17:17:22,931 [main]
> org.apache.cassandra.db.lifecycle.LogTransaction:561 - Unexpected disk state:
> failed to read transaction log
> [nb_txn_stream_39d5f6b0-fb81-11ed-8f46-e97b3f61511e.log in
> /datadir1/keyspace/table-c9527530a0d611e8813f034699fc9043]
> Files and contents follow:
> /datadir1/keyspace/table-c9527530a0d611e8813f034699fc9043/nb_txn_stream_39d5f6b0-fb81-11ed-8f46-e97b3f61511e.log
> ABORT:[,0,0][737437348]
> ***This record should have been the last one in all replicas
>
> ADD:[/datadir1/keyspace/table-c9527530a0d611e8813f034699fc9043/nb-284490-big-,0,8][2493503833]
> {code}
> The root cause is a race during streaming exception handling.
> Although concurrent modification of to the {{LogTransaction}} was added for
> CASSANDRA-16225, there is nothing to prevent usage after the transaction is
> completed (committed/aborted) once it has been processed by
> {{TransactionTidier}} (after the last reference is released). Before the
> transaction is tidied, the {{LogFile}} keeps a list of records that are
> checked for completion before adding new entries. In {{TransactionTidier}}
> {{LogFile.records}} are cleared as no longer needed, however the
> LogTransaction/LogFile is still accessible to the stream.
> The changes in CASSANDRA-17273 added a parallel set of {{onDiskRecords}} that
> could be used to reliably recreate the transaction log at any new datadirs
> the same as the existing
> datadirs - regardless of the effect of
> {{LogTransaction.untrackNew/LogFile.remove}}
> If a streaming exception causes the LogTransaction to be aborted and tidied
> just before {{SimpleSSTableMultiWriter}} calls trackNew to add a new sstable.
> At the time of the call, the {{LogFile}} will not contain any {{LogReplicas}},
> {{LogFile.records}} will be empty, and {{LogFile.onDiskRecords}} will contain
> an {{ABORT}}.
> When {{LogTransaction.trackNew/LogFile.add}} is called, the check for
> completed transaction fails as records is empty, there are no replicas on the
> datadir, so {{maybeCreateReplicas}} creates a new txnlog file replica
> containing ABORT, then
> appends an ADD record.
> The LogFile has already been tidied after the abort so the txnlog file is not
> removed and sits on disk until a restart, causing the faiulre.
> There is a related exception caused with a different interleaving of aborts,
> after an sstable is added, however this is just a nuisance in the logs as the
> LogRelica is already created with an {{ADD}} record first.
> {code:java}
> java.lang.AssertionError:
> [ADD:[/datadir1/keyspace/table/nb-23314378-big-,0,8][1869379820]] is not
> tracked by 55be35b0-35d1-11ee-865d-8b1e3c48ca06
> at org.apache.cassandra.db.lifecycle.LogFile.remove(LogFile.java:388)
> at
> org.apache.cassandra.db.lifecycle.LogTransaction.untrackNew(LogTransaction.java:158)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.untrackNew(LifecycleTransaction.java:577)
> at
> org.apache.cassandra.db.streaming.CassandraStreamReceiver$1.untrackNew(CassandraStreamReceiver.java:149)
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.abort(SimpleSSTableMultiWriter.java:95)
> at
> org.apache.cassandra.io.sstable.format.RangeAwareSSTableWriter.abort(RangeAwareSSTableWriter.java:191)
> at
> org.apache.cassandra.db.streaming.CassandraCompressedStreamReader.read(CassandraCompressedStreamReader.java:115)
> at
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:85)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:53)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:38)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:53)
> at
>
[jira] [Updated] (CASSANDRA-18778) Empty keystore_password no longer allowed on encryption_options
[
https://issues.apache.org/jira/browse/CASSANDRA-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18778:
-
Bug Category: Parent values: Code(13163)
Complexity: Normal
Discovered By: User Report
Fix Version/s: 4.1.x
5.0.x
Severity: Normal
Status: Open (was: Triage Needed)
> Empty keystore_password no longer allowed on encryption_options
> ---
>
> Key: CASSANDRA-18778
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18778
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Normal
> Fix For: 4.1.x, 5.0.x
>
>
> After CASSANDRA-18124 (introduced in 4.1.2 and 5.0) it is no longer possible
> to set an empty {{keystore_password}} under {{client_encryption_options}} or
> {{server_encryption_options}} using the default implementation
> {{{}DefaultSslContextFactory{}}}.
> While keytool does not allow generating keystores with empty passwords, it
> does support reading them. It is not uncommon to use PKCS12 certificates
> generated by other tools (eg. openssl) that do not enforce passwords.
> The fix for this should be pretty straightforward, which should involve
> changing
> [FileBasedSslContextFactory.validatePassword|https://github.com/apache/cassandra/blob/cassandra-4.1.2/src/java/org/apache/cassandra/security/FileBasedSslContextFactory.java#L128-L135]
> to only disallow null passwords (which would be consistent with previous
> versions). I will create pull requests against the relevant branches shortly.
> {noformat}
> Exception (org.apache.cassandra.exceptions.ConfigurationException)
> encountered during startup: Failed to initialize SSL
> org.apache.cassandra.exceptions.ConfigurationException: Failed to initialize
> SSL
> at
> org.apache.cassandra.config.DatabaseDescriptor.applySslContext(DatabaseDescriptor.java:1155)
> at
> org.apache.cassandra.config.DatabaseDescriptor.applyAll(DatabaseDescriptor.java:390)
> at
> org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:204)
> at
> org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:188)
> at
> org.apache.cassandra.service.CassandraDaemon.applyConfig(CassandraDaemon.java:804)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:747)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:875)
> Caused by: java.io.IOException: Failed to create SSL context using Native
> transport
> at
> org.apache.cassandra.security.SSLFactory.validateSslContext(SSLFactory.java:405)
> at
> org.apache.cassandra.config.DatabaseDescriptor.applySslContext(DatabaseDescriptor.java:1150)
> ... 6 more
> Caused by: java.lang.IllegalArgumentException: 'keystore_password' must be
> specified
> at
> org.apache.cassandra.security.FileBasedSslContextFactory.validatePassword(FileBasedSslContextFactory.java:133)
> at
> org.apache.cassandra.security.FileBasedSslContextFactory.buildKeyManagerFactory(FileBasedSslContextFactory.java:151)
> at
> org.apache.cassandra.security.AbstractSslContextFactory.createNettySslContext(AbstractSslContextFactory.java:181)
> at
> org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:168)
> at
> org.apache.cassandra.security.SSLFactory.validateSslContext(SSLFactory.java:355)
> ... 7 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18778) Empty keystore_password no longer allowed on encryption_options
Andy Tolbert created CASSANDRA-18778:
Summary: Empty keystore_password no longer allowed on
encryption_options
Key: CASSANDRA-18778
URL: https://issues.apache.org/jira/browse/CASSANDRA-18778
Project: Cassandra
Issue Type: Bug
Components: Local/Config
Reporter: Andy Tolbert
Assignee: Andy Tolbert
After CASSANDRA-18124 (introduced in 4.1.2 and 5.0) it is no longer possible to
set an empty {{keystore_password}} under {{client_encryption_options}} or
{{server_encryption_options}} using the default implementation
{{{}DefaultSslContextFactory{}}}.
While keytool does not allow generating keystores with empty passwords, it does
support reading them. It is not uncommon to use PKCS12 certificates generated
by other tools (eg. openssl) that do not enforce passwords.
The fix for this should be pretty straightforward, which should involve
changing
[FileBasedSslContextFactory.validatePassword|https://github.com/apache/cassandra/blob/cassandra-4.1.2/src/java/org/apache/cassandra/security/FileBasedSslContextFactory.java#L128-L135]
to only disallow null passwords (which would be consistent with previous
versions). I will create pull requests against the relevant branches shortly.
{noformat}
Exception (org.apache.cassandra.exceptions.ConfigurationException) encountered
during startup: Failed to initialize SSL
org.apache.cassandra.exceptions.ConfigurationException: Failed to initialize SSL
at
org.apache.cassandra.config.DatabaseDescriptor.applySslContext(DatabaseDescriptor.java:1155)
at
org.apache.cassandra.config.DatabaseDescriptor.applyAll(DatabaseDescriptor.java:390)
at
org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:204)
at
org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:188)
at
org.apache.cassandra.service.CassandraDaemon.applyConfig(CassandraDaemon.java:804)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:747)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:875)
Caused by: java.io.IOException: Failed to create SSL context using Native
transport
at
org.apache.cassandra.security.SSLFactory.validateSslContext(SSLFactory.java:405)
at
org.apache.cassandra.config.DatabaseDescriptor.applySslContext(DatabaseDescriptor.java:1150)
... 6 more
Caused by: java.lang.IllegalArgumentException: 'keystore_password' must be
specified
at
org.apache.cassandra.security.FileBasedSslContextFactory.validatePassword(FileBasedSslContextFactory.java:133)
at
org.apache.cassandra.security.FileBasedSslContextFactory.buildKeyManagerFactory(FileBasedSslContextFactory.java:151)
at
org.apache.cassandra.security.AbstractSslContextFactory.createNettySslContext(AbstractSslContextFactory.java:181)
at
org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:168)
at
org.apache.cassandra.security.SSLFactory.validateSslContext(SSLFactory.java:355)
... 7 more
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18360) Test Failure: o.a.c.cql3.validation.operations.AlterTest#testDropListAndAddListWithSameName
[
https://issues.apache.org/jira/browse/CASSANDRA-18360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755693#comment-17755693
]
Brandon Williams commented on CASSANDRA-18360:
--
Hit it again here:
https://app.circleci.com/pipelines/github/driftx/cassandra/1215/workflows/9ac94926-f3b4-4787-b924-798445eb0b48/jobs/46281/tests
but without any variants.
> Test Failure:
> o.a.c.cql3.validation.operations.AlterTest#testDropListAndAddListWithSameName
> ---
>
> Key: CASSANDRA-18360
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18360
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
>
> The unit test
> {{org.apache.cassandra.cql3.validation.operations.AlterTest#testDropListAndAddListWithSameName}}
> is flaky at least on trunk. Flakiness seems lesser than 1%.
> While I haven't seen it on Jenkins yet, it can easily be reproduced on
> CircleCI with the multiplexer:
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2731/workflows/58edc2f6-9a21-4d09-b783-b7fb15e1b320/jobs/32235
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2731/workflows/58edc2f6-9a21-4d09-b783-b7fb15e1b320/jobs/32234
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2731/workflows/739a95d3-8e42-4447-93dd-122fc16fdd7d/jobs/32233/tests
> Those runs show two types of errors:
> {code}
> junit.framework.AssertionFailedError: Invalid value for row 0 column 1
> (content of type text), expected but got
> at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1506)
> at
> org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:102)
> {code}
> and:
> {code}
> org.apache.cassandra.serializers.MarshalException: Invalid UTF-8 bytes
> 00e0279515437f00
> at
> org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:46)
> at
> org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:29)
> at
> org.apache.cassandra.serializers.TypeSerializer.deserialize(TypeSerializer.java:37)
> at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1494)
> at
> org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:102)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The CircleCI config I used to reproduce the test failure can be generated
> with:
> {code}
> .circleci/generate.sh -p \
> -e REPEATED_UTESTS_COUNT=500 \
> -e REPEATED_UTESTS=org.apache.cassandra.cql3.validation.operations.AlterTest
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755689#comment-17755689
]
Brandon Williams commented on CASSANDRA-18710:
--
Actually, adding the flush was probably incorrect since insertN [does
that|https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/io/DiskSpaceMetricsTest.java#L139].
Something must be wrong with the metrics, because if I dig into nightlies for
the
[log|https://nightlies.apache.org/cassandra/trunk/Cassandra-trunk-test-compression/1741/Cassandra-trunk-test-compression/jdk=jdk_17_latest,label=cassandra,split=6/build/test/logs/compression.jdk17/TEST-org.apache.cassandra.io.DiskSpaceMetricsTest.log.xz]
from your last failure, the smallest flush is 4.926KiB.
> Test failure:
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from
> org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
> --
>
> Key: CASSANDRA-18710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18710
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.x
>
>
> Seen here:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1644/testReport/org.apache.cassandra.io/DiskSpaceMetricsTest/testFlushSize__jdk17/]
> h3.
> {code:java}
> Error Message
> expected:<7200.0> but was:<1367.83970468544>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<7200.0> but
> was:<1367.83970468544> at
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize(DiskSpaceMetricsTest.java:119)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18564) Test Failure: MixedModeAvailabilityV3XAllOneTest.testAvailabilityCoordinatorUpgraded
[
https://issues.apache.org/jira/browse/CASSANDRA-18564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755688#comment-17755688
]
Caleb Rackliffe commented on CASSANDRA-18564:
-
Saw this today in a 5.0-based branch:
[https://app.circleci.com/pipelines/github/adelapena/cassandra/3105/workflows/3cf378e2-999f-4d1b-afc4-d517ce0f77e0/jobs/67928/tests#failed-test-0]
> Test Failure:
> MixedModeAvailabilityV3XAllOneTest.testAvailabilityCoordinatorUpgraded
>
>
> Key: CASSANDRA-18564
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18564
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Andres de la Peña
>Priority: Normal
>
> The JVM upgrade dtest
> {{MixedModeAvailabilityV3XAllOneTest.testAvailabilityCoordinatorUpgraded}}
> seems to be flaky at least in {{trunk}}:
> {code}
> junit.framework.AssertionFailedError: Error in test '4.0.11 -> [5.0]' while
> upgrading to '5.0'; successful upgrades []
> at
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:348)
> at
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityTestBase.testAvailability(MixedModeAvailabilityTestBase.java:154)
> at
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityTestBase.testAvailabilityCoordinatorUpgraded(MixedModeAvailabilityTestBase.java:74)
> Caused by: java.lang.AssertionError: Unexpected error while reading in case
> write-read consistency ALL-ONE with upgraded coordinator and 2 nodes down:
> org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
> received only 0 responses.
> at
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityTestBase.lambda$testAvailability$6(MixedModeAvailabilityTestBase.java:145)
> at
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:339)
> Caused by: org.apache.cassandra.exceptions.ReadTimeoutException: Operation
> timed out - received only 0 responses.
> at
> org.apache.cassandra.service.reads.ReadCallback.awaitResults(ReadCallback.java:162)
> at
> org.apache.cassandra.service.reads.AbstractReadExecutor.awaitResponses(AbstractReadExecutor.java:387)
> at
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:2124)
> at
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1995)
> at
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1873)
> at
> org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:1286)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:364)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:293)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:105)
> at
> org.apache.cassandra.distributed.impl.Coordinator.unsafeExecuteInternal(Coordinator.java:122)
> at
> org.apache.cassandra.distributed.impl.Coordinator.unsafeExecuteInternal(Coordinator.java:103)
> at
> org.apache.cassandra.distributed.impl.Coordinator.lambda$executeWithResult$0(Coordinator.java:66)
> at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
> at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:750)
> {code}
> This has failed 143 times in 500 iterations of this CircleCI run:
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2927/workflows/fcd1cd60-826b-484a-8e81-d3ba640f7de9/jobs/47659/tests
> The failure has also recently appeared on Jenkins too:
> *
> https://ci-cassandra.apache.org/job/Cassandra-trunk/1585/testReport/org.apache.cassandra.distributed.upgrade/MixedModeAvailabilityV3XAllOneTest/testAvailabilityCoordinatorUpgraded__jdk11/
> Given that the failure has just appeared on Jenkins and it fails relatively
> easily on CircleCI, it's likely that it has been broken by a very recent
> change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18508) Sensitive JMX SSL configuration options can be easily exposed
[
https://issues.apache.org/jira/browse/CASSANDRA-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-18508:
---
Fix Version/s: 5.0.x
(was: 5.0)
> Sensitive JMX SSL configuration options can be easily exposed
> -
>
> Key: CASSANDRA-18508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18508
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Encryption
>Reporter: Anthony Grasso
>Assignee: Anthony Grasso
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x
>
>
> We need a way to specify sensitive JMX SSL configuration options to avoid
> them being easily exposed.
> When encrypting the JMX connection the passwords for the key and trust stores
> must be specified using the {{javax.net.ssl.keyStorePassword}} and
> {{javax.net.ssl.trustStorePassword}} options respectively in the
> _cassandra-env.sh_ file. After Cassandra is started it is possible to see the
> passwords by looking the running process ({{ps aux | grep "cassandra"}}).
> Java 8 has the ability to specify a configuration file that can contain these
> security sensitive settings using the {{com.sun.management.config.file}}
> argument. However, despite what the documentation
> ([https://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html#gdevf])
> says, both the {{com.sun.management.jmxremote}} and
> {{com.sun.management.jmxremote.port}} arguments need to be defined in the
> _cassandra-env.sh_ for the JVM to read the contents of the file.
> The problem with defining the {{com.sun.management.jmxremote.port}} argument
> is it conflicts with the {{cassandra.jmx.remote.port}} argument. Even if the
> port numbers are different, attempting an encrypted JMX connection using
> {{nodetool}} fails and we see a {{ConnectException: 'Connection refused
> (Connection refused)'}} error.
> One possible way to fix this is to introduce a new option that would allow a
> file to be passed containing the JMX encryption options.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18324) CEP-15: (C*) Head of line blocking of Txn causes large latencies and some Txn to no longer make progress
[ https://issues.apache.org/jira/browse/CASSANDRA-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18324: --- Fix Version/s: 5.0.x (was: 5.0) > CEP-15: (C*) Head of line blocking of Txn causes large latencies and some Txn > to no longer make progress > > > Key: CASSANDRA-18324 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18324 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 5.0.x > > > Accord perf tests discovered an issue where a command becomes stuck in the > Preaccept phase and dependent operations aren't able to make progress -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18736) Streaming exception race creates corrupt transaction log files that prevent restart
[
https://issues.apache.org/jira/browse/CASSANDRA-18736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-18736:
---
Fix Version/s: (was: 5.0)
> Streaming exception race creates corrupt transaction log files that prevent
> restart
> ---
>
> Key: CASSANDRA-18736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18736
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Local/Startup and Shutdown
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x
>
>
> On restart, Cassandra logs this message and terminates.
> {code:java}
> ERROR 2023-07-17T17:17:22,931 [main]
> org.apache.cassandra.db.lifecycle.LogTransaction:561 - Unexpected disk state:
> failed to read transaction log
> [nb_txn_stream_39d5f6b0-fb81-11ed-8f46-e97b3f61511e.log in
> /datadir1/keyspace/table-c9527530a0d611e8813f034699fc9043]
> Files and contents follow:
> /datadir1/keyspace/table-c9527530a0d611e8813f034699fc9043/nb_txn_stream_39d5f6b0-fb81-11ed-8f46-e97b3f61511e.log
> ABORT:[,0,0][737437348]
> ***This record should have been the last one in all replicas
>
> ADD:[/datadir1/keyspace/table-c9527530a0d611e8813f034699fc9043/nb-284490-big-,0,8][2493503833]
> {code}
> The root cause is a race during streaming exception handling.
> Although concurrent modification of to the {{LogTransaction}} was added for
> CASSANDRA-16225, there is nothing to prevent usage after the transaction is
> completed (committed/aborted) once it has been processed by
> {{TransactionTidier}} (after the last reference is released). Before the
> transaction is tidied, the {{LogFile}} keeps a list of records that are
> checked for completion before adding new entries. In {{TransactionTidier}}
> {{LogFile.records}} are cleared as no longer needed, however the
> LogTransaction/LogFile is still accessible to the stream.
> The changes in CASSANDRA-17273 added a parallel set of {{onDiskRecords}} that
> could be used to reliably recreate the transaction log at any new datadirs
> the same as the existing
> datadirs - regardless of the effect of
> {{LogTransaction.untrackNew/LogFile.remove}}
> If a streaming exception causes the LogTransaction to be aborted and tidied
> just before {{SimpleSSTableMultiWriter}} calls trackNew to add a new sstable.
> At the time of the call, the {{LogFile}} will not contain any {{LogReplicas}},
> {{LogFile.records}} will be empty, and {{LogFile.onDiskRecords}} will contain
> an {{ABORT}}.
> When {{LogTransaction.trackNew/LogFile.add}} is called, the check for
> completed transaction fails as records is empty, there are no replicas on the
> datadir, so {{maybeCreateReplicas}} creates a new txnlog file replica
> containing ABORT, then
> appends an ADD record.
> The LogFile has already been tidied after the abort so the txnlog file is not
> removed and sits on disk until a restart, causing the faiulre.
> There is a related exception caused with a different interleaving of aborts,
> after an sstable is added, however this is just a nuisance in the logs as the
> LogRelica is already created with an {{ADD}} record first.
> {code:java}
> java.lang.AssertionError:
> [ADD:[/datadir1/keyspace/table/nb-23314378-big-,0,8][1869379820]] is not
> tracked by 55be35b0-35d1-11ee-865d-8b1e3c48ca06
> at org.apache.cassandra.db.lifecycle.LogFile.remove(LogFile.java:388)
> at
> org.apache.cassandra.db.lifecycle.LogTransaction.untrackNew(LogTransaction.java:158)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.untrackNew(LifecycleTransaction.java:577)
> at
> org.apache.cassandra.db.streaming.CassandraStreamReceiver$1.untrackNew(CassandraStreamReceiver.java:149)
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.abort(SimpleSSTableMultiWriter.java:95)
> at
> org.apache.cassandra.io.sstable.format.RangeAwareSSTableWriter.abort(RangeAwareSSTableWriter.java:191)
> at
> org.apache.cassandra.db.streaming.CassandraCompressedStreamReader.read(CassandraCompressedStreamReader.java:115)
> at
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:85)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:53)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:38)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:53)
> at
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:172)
[jira] [Updated] (CASSANDRA-18278) Add a tool to clean redundant data for native secondary index
[
https://issues.apache.org/jira/browse/CASSANDRA-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-18278:
---
Fix Version/s: 5.x
(was: 5.0)
> Add a tool to clean redundant data for native secondary index
> ---
>
> Key: CASSANDRA-18278
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18278
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/2i Index, Tool/nodetool
>Reporter: Maxwell Guo
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
>
> As we know Cassandra' secondary index is a local secondary index , and for
> every data update , and the every update hit the indexed columns. The old
> redundant data (Stale Entries) for index table are keeped in the table only
> when the data are read (may be a little like read repair ).
> So there may exist some old and useless data for index table if they are not
> read. So we would like to support a tool that can remove the old useless data
> .See the picture below , we create a table with a secondary index on c1
> column , then update data with same pk ,different c1 value, and we flush
> after every update, after that we force a major on the index table . See the
> sstable dump for secondary index (The dump tool for secondary index can not
> be used but fortunately we use the
> [CASSANDRA-17698|https://issues.apache.org/jira/browse/CASSANDRA-17698])， and
> we can see the content of index sstable.
> Below are the cql and dump result.
> {code:java}
> cqlsh> DESC ks.tb
> CREATE TABLE ks.tb (
> pk int PRIMARY KEY,
> c1 int
> ) WITH additional_write_policy = '99p'
> AND allow_auto_snapshot = true
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND cdc = false
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '16', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND memtable = 'default'
> AND crc_check_chance = 1.0
> AND default_time_to_live = 0
> AND extensions = {}
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair = 'BLOCKING'
> AND speculative_retry = '99p';
> CREATE INDEX idx ON ks.tb (c1);
> cqlsh> INSERT INTO ks.tb(pk, c1)values (1, 1);
> cqlsh> INSERT INTO ks.tb(pk, c1)values (1, 2);
> cqlsh> INSERT INTO ks.tb(pk, c1)values (1, 3);
> cqlsh>
> {code}
> On the other hand we flush after every update and force a major at the end.
> {code:java}
> bin git:(trunk) ✗ ./nodetool flush
> ➜ bin git:(trunk) ✗ ./nodetool flush
> ➜ bin git:(trunk) ✗ ./nodetool flush
> ➜ bin git:(trunk) ✗ ./nodetool compact ks tb.idx
> ➜ bin git:(trunk) ✗ ../tools/bin/sstabledump
> ../data/data/ks/tb-65d902b0b2bc11ed86ed81daebeca99d/.idx/nb-13-big-Data.db
> [
> {
> "table kind" : "INDEX",
> "partition" : {
> "key" : [ "1" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "clustering" : [ 1 ],
> "liveness_info" : { "tstamp" : "2023-02-23T03:21:57.638558Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "table kind" : "INDEX",
> "partition" : {
> "key" : [ "2" ],
> "position" : 29
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 47,
> "clustering" : [ 1 ],
> "liveness_info" : { "tstamp" : "2023-02-23T03:22:19.834466Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "table kind" : "INDEX",
> "partition" : {
> "key" : [ "3" ],
> "position" : 61
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 79,
> "clustering" : [ 1 ],
> "liveness_info" : { "tstamp" : "2023-02-23T03:22:27.532174Z" },
> "cells" : [ ]
> }
> ]
> }
> ]%
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18733) Waiting indefinitely on ReceivedMessage response in StreamSession#receive() can cause deadlock
[
https://issues.apache.org/jira/browse/CASSANDRA-18733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-18733:
---
Fix Version/s: (was: 5.0)
> Waiting indefinitely on ReceivedMessage response in StreamSession#receive()
> can cause deadlock
> --
>
> Key: CASSANDRA-18733
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18733
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Consistency/Streaming
>Reporter: Caleb Rackliffe
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
>
> I've observed in a recent stack trace from a node running 4.1 what looks like
> a deadlock around the {{StreamSession}} monitor lock when
> {{StreamSession#receive()}} waits via {{syncUninteruptibly()}} for a response
> to a control message.
> {noformat}
> "Messaging-EventLoop-3-10" #320 daemon prio=5 os_prio=0 cpu=57979617.98ms
> elapsed=5587916.03s tid=0x7f056e88ae00 nid=0x80ec waiting for monitor
> entry [0x7f056d277000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:524)
> - waiting to lock <0x0006816fae70> (a
> org.apache.cassandra.streaming.StreamSession)
> at
> org.apache.cassandra.streaming.StreamSession.onError(StreamSession.java:690)
> at
> org.apache.cassandra.streaming.async.StreamingMultiplexedChannel.onMessageComplete(StreamingMultiplexedChannel.java:264)
> at
> org.apache.cassandra.streaming.async.StreamingMultiplexedChannel.lambda$sendMessage$1(StreamingMultiplexedChannel.java:233)
> at
> org.apache.cassandra.streaming.async.StreamingMultiplexedChannel$$Lambda$2029/0x0008007a0c40.operationComplete(Unknown
> Source)
> at
> org.apache.cassandra.utils.concurrent.ListenerList.notifyListener(ListenerList.java:134)
> at
> org.apache.cassandra.utils.concurrent.ListenerList.notifyListener(ListenerList.java:148)
> at
> org.apache.cassandra.utils.concurrent.ListenerList$GenericFutureListenerList.notifySelf(ListenerList.java:190)
> at
> org.apache.cassandra.utils.concurrent.ListenerList.lambda$notifyExclusive$0(ListenerList.java:124)
> at
> org.apache.cassandra.utils.concurrent.ListenerList$$Lambda$950/0x000800666040.accept(Unknown
> Source)
> at
> org.apache.cassandra.utils.concurrent.IntrusiveStack.forEach(IntrusiveStack.java:195)
> at
> org.apache.cassandra.utils.concurrent.ListenerList.notifyExclusive(ListenerList.java:124)
> at
> org.apache.cassandra.utils.concurrent.ListenerList.notify(ListenerList.java:96)
> at
> org.apache.cassandra.utils.concurrent.AsyncFuture.trySet(AsyncFuture.java:104)
> at
> org.apache.cassandra.utils.concurrent.AbstractFuture.tryFailure(AbstractFuture.java:148)
> at
> org.apache.cassandra.utils.concurrent.AsyncPromise.tryFailure(AsyncPromise.java:139)
> at
> io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:1009)
> at
> io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:870)
> at
> io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1367)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:764)
> at
> io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1071)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(java.base@11.0.16/Thread.java:829)
> {noformat}
> It seems that while {{receive()} is holding the monitor lock on
> {{StreamSession}}, the callback that executes on a different thread for the
> control message it sends carries an error. This error, when handled in
> {{onError()}}, then calls {{closeSession()}}, which tries to acquire the
> monitor lock already held in {{receive()}}.
> {noformat}
>
[jira] [Updated] (CASSANDRA-18722) Support Dynamic Port Allocation for in-jvm dtest framework
[
https://issues.apache.org/jira/browse/CASSANDRA-18722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-18722:
---
Fix Version/s: 4.0.x
(was: 4.0)
(was: 5.0)
> Support Dynamic Port Allocation for in-jvm dtest framework
> --
>
> Key: CASSANDRA-18722
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18722
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently, {{INodeProvisionStrategy}} supports two strategies
> {{OneNetworkInterface}} and {{MultipleNetworkInterfaces}}. However the
> {{seedPort}}, {{storagePorts}}, {{nativeTransportPorts}}, and {{jmxPorts}}
> are always fixed or a function of the node number.
> In order to better support parallel test runs, we need to support dynamic
> port allocation for the {{seedPort}}, {{storagePorts}},
> {{nativeTransportPorts}}, and {{jmxPorts}}. This would enable us to more
> easily write tests that can run in parallel. This effort is only a stepping
> stone in what's required to run more tests in parallel, but it allows us to
> begin somewhere with the in-jvm dtest framework.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16325) Update streaming metrics incrementally
[ https://issues.apache.org/jira/browse/CASSANDRA-16325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16325: --- Fix Version/s: (was: 5.0) > Update streaming metrics incrementally > -- > > Key: CASSANDRA-16325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16325 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Metrics >Reporter: Paulo Motta >Assignee: Isaac Reath >Priority: Normal > Labels: lhf > Fix For: 5.0.x > > Time Spent: 10h 50m > Remaining Estimate: 0h > > Currently the inbound and outbound streamed bytes metrics are incremented > after each file is streamed, what doesn't represent the current number of > bytes streamed since it can take a long time for a large file to be streamed. > We should update the metric incrementally as data is streamed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-11745) Add bytes limit to queries and paging
[ https://issues.apache.org/jira/browse/CASSANDRA-11745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-11745: --- Fix Version/s: (was: 5.0) > Add bytes limit to queries and paging > - > > Key: CASSANDRA-11745 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11745 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Richard Low >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 5.0.x > > Time Spent: 20m > Remaining Estimate: 0h > > For some data models, values may be of very different sizes. When querying > data, limit by count doesn’t work well and leads to timeouts. It would be > much better to limit by size of the response, probably by stopping at the > first row that goes above the limit. This applies to paging too so you can > safely page through such data without timeout worries. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18673) Reduce size of per-SSTable index components
[ https://issues.apache.org/jira/browse/CASSANDRA-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18673: --- Fix Version/s: 5.x (was: 5.0) (was: 5.1) > Reduce size of per-SSTable index components > --- > > Key: CASSANDRA-18673 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18673 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0.x, 5.x > > Time Spent: 6.5h > Remaining Estimate: 0h > > The current per-SSTable index components are large because the primary keys > that are stored in them include the token as part of the byte comparable. The > byte comparable puts the token first meaning that we get very little prefix > compression from either the trie or the sorted terms store. > We can fix this by removing the token from the primary key serialization. > This would allow us to get the prefix compression from the trie and the > sorted terms store. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18444) CEP-15: (C*) Transactional Metadata Integration
[ https://issues.apache.org/jira/browse/CASSANDRA-18444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18444: --- Fix Version/s: (was: 5.0) > CEP-15: (C*) Transactional Metadata Integration > --- > > Key: CASSANDRA-18444 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18444 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 5.0.x > > > Integration transactional metadata with accord. TCM should update Accord > topology and schema, and Accord epochs should map to tcm epochs -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18078) Consider removing MAXWRITETIME function
[
https://issues.apache.org/jira/browse/CASSANDRA-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-18078:
---
Fix Version/s: (was: 5.0)
> Consider removing MAXWRITETIME function
> ---
>
> Key: CASSANDRA-18078
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18078
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Syntax
>Reporter: Andres de la Peña
>Priority: Normal
> Fix For: 5.0.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> CASSANDRA-17425 added a new {{MAXWRITETIME}} function that allows to retrieve
> the maximum timestamp of a multi-cell column. For example:
> {code:java}
> > CREATE TABLE t (k int PRIMARY KEY, v set);
> > INSERT INTO t (k, v) VALUES (1, {1, 2}) USING TIMESTAMP 100;
> > UPDATE t USING TIMESTAMP 200 SET v += {3} WHERE k=1;
> > SELECT maxwritetime(v) FROM t;
> maxwritetime(v)
> -
> 200
> {code}
> Later, CASSANDRA-8877 added the means to retrieve the write times and TTLs of
> each of the cells in a multi-cell column:
> {code:java}
> > SELECT writetime(v) FROM t;
> writetime(v)
> -
> [100, 100, 200]
> > SELECT writetime(v[1]) FROM t;
> writetime(v[1])
> -
> 100
> {code}
> Quite recently, CASSANDRA-18060 has added generic CQL functions to get the
> min and max items in a collection. Those functions can be used in combination
> with the classic {{writetime}} function to get the same results as the new
> {{maxwritetime}} function:
> {code:java}
> > SELECT collection_max(writetime(v)) FROM t;
> system.collection_max(writetime(v))
> -
> 200
> {code}
> Those new functions can also be used to get the min timestamp, or the min/max
> TTL, for which there isn't a specific function:
> {code:java}
> SELECT collection_min(writetime(v)) FROM t;
> SELECT collection_max(writetime(v)) FROM t;
> SELECT collection_avg(writetime(v)) FROM t;
> SELECT collection_min(ttl(v)) FROM t;
> SELECT collection_max(ttl(v)) FROM t;
> SELECT collection_avg(ttl(v)) FROM t;
> {code}
> I think this makes the new {{maxwritetime}} mostly redundant, since the new
> functions can achieve the same results in a more generic way. Since the new
> {{maxwritetime}} function is only on trunk, we should consider whether we
> want to remove it in favour of the generic functions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18745) cqlsh should warn on server version mismatch
[ https://issues.apache.org/jira/browse/CASSANDRA-18745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755670#comment-17755670 ] Brandon Williams edited comment on CASSANDRA-18745 at 8/17/23 7:38 PM: --- More or less as described above: ||Branch||CI|| |[3.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-3.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1214/workflows/247dc95c-1bd8-4670-ac0e-9c33fb4343bc]| |[3.11|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-3.11]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1217/workflows/32e0426b-3436-4b49-a7d4-dd33fa9c3c51]| |[4.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-4.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1219/workflows/74ff343f-636c-417f-bfd1-14c14185a71c], [j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1219/workflows/ad6b56c2-4a87-4c7f-97ac-480286aadc18]| |[4.1|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-4.1]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1216/workflows/9e870d59-5fd3-4921-882a-a85f01f5a944], [j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1216/workflows/9caa1fcc-8894-47bc-8d46-3197202d1ffd]| |[5.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-5.0]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1218/workflows/21c140f1-3f3d-4e2f-864f-e2aeaabcd4e5], [j17|https://app.circleci.com/pipelines/github/driftx/cassandra/1218/workflows/fddb8227-5289-437b-9b49-506da34cdb3e]| |[trunk|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-trunk]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1215/workflows/ecb0cebd-2bc7-408e-b037-27559ab3e7e5], [j17|https://app.circleci.com/pipelines/github/driftx/cassandra/1215/workflows/9ac94926-f3b4-4787-b924-798445eb0b48]| was (Author: brandon.williams): More or less as described above: ||Branch||CI|| |[3.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-3.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1210/workflows/c6385410-2088-4bbd-9b10-89d8075c7b62]| |[3.11|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-3.11]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1209/workflows/f2fa04bc-0409-47ca-84a0-d0465f75f44d]| |[4.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-4.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1211/workflows/32845033-f942-468b-a7f7-1a2c4686fb69], [j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1211/workflows/cc100a27-239e-479a-9dc7-257ae0a5123b]| |[4.1|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-4.1]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1206/workflows/8d29a73a-0d87-478c-aa0a-b1706e726289], [j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1206/workflows/81e7cff4-fffa-4a0f-a992-0814eb1b7f18]| |[5.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-5.0]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1213/workflows/1a76f01e-e23e-40a4-a4a5-abdd8232e7db], [j17|https://app.circleci.com/pipelines/github/driftx/cassandra/1213/workflows/601e889c-0a5d-4753-a1c1-998239563fa4]| |[trunk|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-trunk]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1212/workflows/500cf7ff-c66a-441f-8bdc-eaa11eba80e2], [j17|https://app.circleci.com/pipelines/github/driftx/cassandra/1212/workflows/07fe8103-c0fc-4448-8cea-089644282560]| > cqlsh should warn on server version mismatch > > > Key: CASSANDRA-18745 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18745 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > > When cqlsh is used against a different version of the server than the one > that it shipped with, it should emit a warning so that users aren't confused > when some things don't work correctly. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18745) cqlsh should warn on server version mismatch
[ https://issues.apache.org/jira/browse/CASSANDRA-18745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams reassigned CASSANDRA-18745: Assignee: Brandon Williams > cqlsh should warn on server version mismatch > > > Key: CASSANDRA-18745 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18745 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > > When cqlsh is used against a different version of the server than the one > that it shipped with, it should emit a warning so that users aren't confused > when some things don't work correctly. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18777) Revise unused project dependencies
[
https://issues.apache.org/jira/browse/CASSANDRA-18777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated CASSANDRA-18777:
Fix Version/s: 5.x
> Revise unused project dependencies
> --
>
> Key: CASSANDRA-18777
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18777
> Project: Cassandra
> Issue Type: Task
>Reporter: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.x
>
>
> There are some project dependencies that are no longer used, but they are
> still declared in the parent pom. I think we can revise the project
> dependencies and clean up the ones that are not used.
> There are two issues here:
> - deleting unused dependencies - it should be agreed by the community in
> advance;
> - adding an automation check to prevent the same problem in the future;
> As a naive approach to finding the list of unused dependencies, the {{mvn
> dependency:analyze}} command can be used. The command will not outline the
> list of dependencies to remove, as some of them may only be used at runtime,
> so the list should be considered cautiously. We put the built classes in the
> {{build/classes}} and {{build/test/classes}} directories, which won't work
> for the maven dependency plugin because it expects the built classes to be in
> the {{target/classes}} directory - to trick that, we can copy them all there
> and run the plugin against the built pom file. Below you can find commands to
> reproduce the result and the result itself.
> h3. The command
> Tested on the latest trunk (commons-codec is there).
> {code}
> # copy packages to the build/target/classes to run maven
> for d in `find ./build/classes -maxdepth 2 -type d -name org -or -name com`;
> do `rsync -a $d ./build/target/classes`; done
> mvn -f ./build/apache-cassandra-5.0-SNAPSHOT.pom
> org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze -Dverbose=true
> -DignoreNonCompile
> {code}
> h3. The result
> {code}
> [WARNING] Used undeclared dependencies found:
> [WARNING]io.netty:netty-handler:jar:4.1.96.Final:compile
> [WARNING]io.netty:netty-common:jar:4.1.96.Final:compile
> [WARNING]io.netty:netty-buffer:jar:4.1.96.Final:compile
> [WARNING]io.netty:netty-codec:jar:4.1.96.Final:compile
> [WARNING]org.checkerframework:checker-qual:jar:3.10.0:compile
> [WARNING]io.netty:netty-transport:jar:4.1.96.Final:compile
> [WARNING]
> io.netty:netty-transport-native-unix-common:jar:4.1.96.Final:compile
> [WARNING]io.netty:netty-transport-classes-epoll:jar:4.1.96.Final:compile
> [WARNING] Unused declared dependencies found:
> [WARNING]com.github.jnr:jffi:jar:native:1.3.11:compile
> [WARNING]net.openhft:affinity:jar:3.23.3:compile
> [WARNING]org.slf4j:log4j-over-slf4j:jar:1.7.25:compile
> [WARNING]io.netty:netty-transport-native-epoll:jar:4.1.94.Final:compile
> [WARNING]com.github.jnr:jnr-constants:jar:0.10.4:compile
> [WARNING]io.dropwizard.metrics:metrics-logback:jar:3.1.5:compile
> [WARNING]org.antlr:ST4:jar:4.0.8:compile
> [WARNING]org.gridkit.jvmtool:mxdump:jar:0.14:compile
> [WARNING]io.netty:netty-tcnative-boringssl-static:jar:2.0.61.Final:compile
> [WARNING]com.google.j2objc:j2objc-annotations:jar:1.3:compile
> [WARNING]org.gridkit.lab:jvm-attach-api:jar:1.5:compile
> [WARNING]org.gridkit.jvmtool:sjk-json:jar:0.14:compile
> [WARNING]org.slf4j:jcl-over-slf4j:jar:1.7.25:compile
> [WARNING]com.github.jnr:jnr-ffi:jar:2.2.13:compile
> [WARNING]commons-codec:commons-codec:jar:1.9:compile
> [WARNING]org.caffinitas.ohc:ohc-core-j8:jar:0.5.1:compile
> [WARNING]org.bouncycastle:bcpkix-jdk15on:jar:1.70:compile
> [WARNING]net.java.dev.jna:jna-platform:jar:5.13.0:compile
> [WARNING]com.github.jnr:jffi:jar:1.3.11:compile
> [WARNING]io.netty:netty-all:jar:4.1.96.Final:compile
> [WARNING]org.gridkit.jvmtool:sjk-stacktrace:jar:0.14:compile
> [WARNING]
> io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.94.Final:compile
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18772) Remove dependency on commons-codec
[ https://issues.apache.org/jira/browse/CASSANDRA-18772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755673#comment-17755673 ] Maxim Muzafarov commented on CASSANDRA-18772: - [~e.dimitrova] I've created the issue: CASSANDRA-18777 > Remove dependency on commons-codec > -- > > Key: CASSANDRA-18772 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18772 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 5.0.x, 5.x > > > The dependency doesn't seem to have any direct usage in our codebase. > The only reference to how it might be used was found in CASSANDRA-12790. > Considering we also deprecate in 5.0 and remove metrics-reporter-config as > part of CASSANDRA-18743, I do not see a reason not to remove this one too. > We should probably take the same approach as CASSANDRA-18743 - deprecate in > 5.0 and remove in 5.1. ML thread is to be opened too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18777) Revise unused project dependencies
Maxim Muzafarov created CASSANDRA-18777:
---
Summary: Revise unused project dependencies
Key: CASSANDRA-18777
URL: https://issues.apache.org/jira/browse/CASSANDRA-18777
Project: Cassandra
Issue Type: Task
Reporter: Maxim Muzafarov
There are some project dependencies that are no longer used, but they are still
declared in the parent pom. I think we can revise the project dependencies and
clean up the ones that are not used.
There are two issues here:
- deleting unused dependencies - it should be agreed by the community in
advance;
- adding an automation check to prevent the same problem in the future;
As a naive approach to finding the list of unused dependencies, the {{mvn
dependency:analyze}} command can be used. The command will not outline the list
of dependencies to remove, as some of them may only be used at runtime, so the
list should be considered cautiously. We put the built classes in the
{{build/classes}} and {{build/test/classes}} directories, which won't work for
the maven dependency plugin because it expects the built classes to be in the
{{target/classes}} directory - to trick that, we can copy them all there and
run the plugin against the built pom file. Below you can find commands to
reproduce the result and the result itself.
h3. The command
Tested on the latest trunk (commons-codec is there).
{code}
# copy packages to the build/target/classes to run maven
for d in `find ./build/classes -maxdepth 2 -type d -name org -or -name com`; do
`rsync -a $d ./build/target/classes`; done
mvn -f ./build/apache-cassandra-5.0-SNAPSHOT.pom
org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze -Dverbose=true
-DignoreNonCompile
{code}
h3. The result
{code}
[WARNING] Used undeclared dependencies found:
[WARNING]io.netty:netty-handler:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-common:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-buffer:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-codec:jar:4.1.96.Final:compile
[WARNING]org.checkerframework:checker-qual:jar:3.10.0:compile
[WARNING]io.netty:netty-transport:jar:4.1.96.Final:compile
[WARNING]
io.netty:netty-transport-native-unix-common:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-transport-classes-epoll:jar:4.1.96.Final:compile
[WARNING] Unused declared dependencies found:
[WARNING]com.github.jnr:jffi:jar:native:1.3.11:compile
[WARNING]net.openhft:affinity:jar:3.23.3:compile
[WARNING]org.slf4j:log4j-over-slf4j:jar:1.7.25:compile
[WARNING]io.netty:netty-transport-native-epoll:jar:4.1.94.Final:compile
[WARNING]com.github.jnr:jnr-constants:jar:0.10.4:compile
[WARNING]io.dropwizard.metrics:metrics-logback:jar:3.1.5:compile
[WARNING]org.antlr:ST4:jar:4.0.8:compile
[WARNING]org.gridkit.jvmtool:mxdump:jar:0.14:compile
[WARNING]io.netty:netty-tcnative-boringssl-static:jar:2.0.61.Final:compile
[WARNING]com.google.j2objc:j2objc-annotations:jar:1.3:compile
[WARNING]org.gridkit.lab:jvm-attach-api:jar:1.5:compile
[WARNING]org.gridkit.jvmtool:sjk-json:jar:0.14:compile
[WARNING]org.slf4j:jcl-over-slf4j:jar:1.7.25:compile
[WARNING]com.github.jnr:jnr-ffi:jar:2.2.13:compile
[WARNING]commons-codec:commons-codec:jar:1.9:compile
[WARNING]org.caffinitas.ohc:ohc-core-j8:jar:0.5.1:compile
[WARNING]org.bouncycastle:bcpkix-jdk15on:jar:1.70:compile
[WARNING]net.java.dev.jna:jna-platform:jar:5.13.0:compile
[WARNING]com.github.jnr:jffi:jar:1.3.11:compile
[WARNING]io.netty:netty-all:jar:4.1.96.Final:compile
[WARNING]org.gridkit.jvmtool:sjk-stacktrace:jar:0.14:compile
[WARNING]
io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.94.Final:compile
[INFO]
[INFO] BUILD SUCCESS
[INFO]
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18745) cqlsh should warn on server version mismatch
[ https://issues.apache.org/jira/browse/CASSANDRA-18745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755670#comment-17755670 ] Brandon Williams edited comment on CASSANDRA-18745 at 8/17/23 7:00 PM: --- More or less as described above: ||Branch||CI|| |[3.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-3.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1210/workflows/c6385410-2088-4bbd-9b10-89d8075c7b62]| |[3.11|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-3.11]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1209/workflows/f2fa04bc-0409-47ca-84a0-d0465f75f44d]| |[4.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-4.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1211/workflows/32845033-f942-468b-a7f7-1a2c4686fb69], [j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1211/workflows/cc100a27-239e-479a-9dc7-257ae0a5123b]| |[4.1|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-4.1]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1206/workflows/8d29a73a-0d87-478c-aa0a-b1706e726289], [j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1206/workflows/81e7cff4-fffa-4a0f-a992-0814eb1b7f18]| |[5.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-5.0]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1207/workflows/1fd29c0a-3a00-464d-98d3-809817572419], [j17|https://app.circleci.com/pipelines/github/driftx/cassandra/1207/workflows/3999cad1-31bd-426d-be0f-31458b7b9b9d]| |[trunk|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-trunk]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1208/workflows/70c6f168-4ac0-4b5b-92b7-5ab90916d9ff], [j17|https://app.circleci.com/pipelines/github/driftx/cassandra/1208/workflows/7efadf51-68ea-4313-a0a1-3c185f4082fa]| was (Author: brandon.williams): More or less as described above: ||Branch||CI|| |[3.11|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-3.11]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1209/workflows/f2fa04bc-0409-47ca-84a0-d0465f75f44d]| |[4.1|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-4.1]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1206/workflows/8d29a73a-0d87-478c-aa0a-b1706e726289], [j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1206/workflows/81e7cff4-fffa-4a0f-a992-0814eb1b7f18]| |[5.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-5.0]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1207/workflows/1fd29c0a-3a00-464d-98d3-809817572419], [j17|https://app.circleci.com/pipelines/github/driftx/cassandra/1207/workflows/3999cad1-31bd-426d-be0f-31458b7b9b9d]| |[trunk|https://github.com/driftx/cassandra/tree/CASSANDRA-18745-trunk]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1208/workflows/70c6f168-4ac0-4b5b-92b7-5ab90916d9ff], [j17|https://app.circleci.com/pipelines/github/driftx/cassandra/1208/workflows/7efadf51-68ea-4313-a0a1-3c185f4082fa]| > cqlsh should warn on server version mismatch > > > Key: CASSANDRA-18745 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18745 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > > When cqlsh is used against a different version of the server than the one > that it shipped with, it should emit a warning so that users aren't confused > when some things don't work correctly. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (7a90dc37 -> 0e7fe18e)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 7a90dc37 generate docs for cceb2c06 new 0e7fe18e generate docs for cceb2c06 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (7a90dc37) \ N -- N -- N refs/heads/asf-staging (0e7fe18e) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4852435 -> 4852435 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[
https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755652#comment-17755652
]
shylaja kokoori commented on CASSANDRA-18688:
-
Mick, to answer your questions
{quote}How difficult is it to edit the bin/cassandra script (or
cassandra.in.sh) for them to get what they want ?
{quote}
Like, Ekaterina said, currently we are allowing versions 11, 17 and anything
above. With current patch, server bring up fails if not a supported version.
bin/cassandra.in.sh changes are not difficult, but there is a bit of learning
curve in my opinion, if one is new to the code.
{quote}What signal (advertently or not) are we giving by providing the `-J`
option ? Would a env var be a better signal ? (I think the use of `-J`, if we
accept it, should still print a loud warning.)
{quote}
-J option is inline with -R, f etc. we already provide for bin/cassandra. By
env var, I assume you mean something like what we had to enable JDK11 in 4.x,
with a -D. This maybe a better option perhaps, if we want to do something
similar with scripts in tools folder. Regardless, I feel providing an option
from command line will be convenient. I agree, with everyone the message needs
to be louder than what I have.
> Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out
> of that
>
>
> Key: CASSANDRA-18688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18688
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Ekaterina Dimitrova
>Assignee: shylaja kokoori
>Priority: Normal
> Fix For: 5.0.x
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently, we limit our users from building with non-default Java versions in
> build.xml.
> They can easily hack build.xml for test purposes with different versions.
> Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit
> people to those two, but only to everything >= 11. We should also put an
> upper limit of 17 in our Cassandra startup scripts. We can also add a flag to
> opt-out if someone wants to test with newer versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18762) Repair triggers OOM with direct buffer memory
[
https://issues.apache.org/jira/browse/CASSANDRA-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755650#comment-17755650
]
Brad Schoening commented on CASSANDRA-18762:
We're going to test this change in 4.0.7:
4.0.7
* Fix multiple BufferPool bugs (CASSANDRA-16681)
> Repair triggers OOM with direct buffer memory
> -
>
> Key: CASSANDRA-18762
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18762
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair
>Reporter: Brad Schoening
>Priority: Normal
> Labels: OutOfMemoryError
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Attachments: Cluster-dm-metrics-1.PNG
>
>
> We are seeing repeated failures of nodes with 16GB of heap and the same size
> (16GB) for direct memory (derived from -Xms). This seems to be related to
> CASSANDRA-15202 which moved merkel trees off-heap in 4.0. Using Cassandra
> 4.0.6.
> {noformat}
> 2023-08-09 04:30:57,470 [INFO ] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 RepairSession.java:202 - [repair
> #5e55a3b0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_a from
> /169.102.200.241:7000
> 2023-08-09 04:30:57,567 [INFO ] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 RepairSession.java:202 - [repair
> #5e0d2900-366d-11ee-a644-d91df26add5e] Received merkle tree for table_b from
> /169.93.192.29:7000
> 2023-08-09 04:30:57,568 [INFO ] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 RepairSession.java:202 - [repair
> #5e1dcad0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_c from
> /169.104.171.134:7000
> 2023-08-09 04:30:57,591 [INFO ] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 RepairSession.java:202 - [repair
> #5e69a0e0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_b from
> /169.79.232.67:7000
> 2023-08-09 04:30:57,876 [INFO ] [Service Thread] cluster_id=101
> ip_address=169.0.0.1 GCInspector.java:294 - G1 Old Generation GC in 282ms.
> Compressed Class Space: 8444560 -> 8372152; G1 Eden Space: 7809794048 -> 0;
> G1 Old Gen: 1453478400 -> 820942800; G1 Survivor Space: 419430400 -> 0;
> Metaspace: 80411136 -> 80176528
> 2023-08-09 04:30:58,387 [ERROR] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 JVMStabilityInspector.java:102 - OutOfMemory error
> letting the JVM handle the error:
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
> at java.base/java.nio.DirectByteBuffer.(DirectByteBuffer.java:118)
> at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:318)
> at org.apache.cassandra.utils.MerkleTree.allocate(MerkleTree.java:742)
> at
> org.apache.cassandra.utils.MerkleTree.deserializeOffHeap(MerkleTree.java:780)
> at org.apache.cassandra.utils.MerkleTree.deserializeTree(MerkleTree.java:751)
> at org.apache.cassandra.utils.MerkleTree.deserialize(MerkleTree.java:720)
> at org.apache.cassandra.utils.MerkleTree.deserialize(MerkleTree.java:698)
> at
> org.apache.cassandra.utils.MerkleTrees$MerkleTreesSerializer.deserialize(MerkleTrees.java:416)
> at
> org.apache.cassandra.repair.messages.ValidationResponse$1.deserialize(ValidationResponse.java:100)
> at
> org.apache.cassandra.repair.messages.ValidationResponse$1.deserialize(ValidationResponse.java:84)
> at
> org.apache.cassandra.net.Message$Serializer.deserializePost40(Message.java:782)
> at org.apache.cassandra.net.Message$Serializer.deserialize(Message.java:642)
> at
> org.apache.cassandra.net.InboundMessageHandler$LargeMessage.deserialize(InboundMessageHandler.java:364)
> at
> org.apache.cassandra.net.InboundMessageHandler$LargeMessage.access$1100(InboundMessageHandler.java:317)
> at
> org.apache.cassandra.net.InboundMessageHandler$ProcessLargeMessage.provideMessage(InboundMessageHandler.java:504)
> at
> org.apache.cassandra.net.InboundMessageHandler$ProcessMessage.run(InboundMessageHandler.java:429)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)no* further _formatting_ is
> done here{noformat}
>
> -XX:+AlwaysPreTouch
> -XX:+CrashOnOutOfMemoryError
> -XX:+ExitOnOutOfMemoryError
> -XX:+HeapDumpOnOutOfMemoryError
> -XX:+ParallelRefProcEnabled
> -XX:+PerfDisableSharedMem
> -XX:+ResizeTLAB
> -XX:+UseG1GC
> -XX:+UseNUMA
> -XX:+UseTLAB
> -XX:+UseThreadPriorities
>
[jira] [Commented] (CASSANDRA-18228) Reorganize Apache C* docs in trunk branch to match the IA
[ https://issues.apache.org/jira/browse/CASSANDRA-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755646#comment-17755646 ] Andres de la Peña commented on CASSANDRA-18228: --- I'll fix it on CASSANDRA-18776 > Reorganize Apache C* docs in trunk branch to match the IA > - > > Key: CASSANDRA-18228 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18228 > Project: Cassandra > Issue Type: New Feature > Components: Documentation >Reporter: Lorina Poland >Assignee: Lorina Poland >Priority: Normal > Fix For: 5.0 > > > An agreed-upon Information Architecture will be used to reorganize the C* 5.0 > docs. > IA doc: > https://docs.google.com/document/d/1A96K73vj9MbJoD7wJNgIKWrOkLq-ZL2cNZAEXSWrciY/edit?usp=sharing -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18776) Restore accidentally deleted documentation for DDM
[ https://issues.apache.org/jira/browse/CASSANDRA-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-18776: -- Bug Category: Parent values: Documentation(13562) Complexity: Low Hanging Fruit Discovered By: Code Inspection Fix Version/s: 5.x Severity: Normal Status: Open (was: Triage Needed) > Restore accidentally deleted documentation for DDM > -- > > Key: CASSANDRA-18776 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18776 > Project: Cassandra > Issue Type: Bug > Components: Documentation >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 5.x > > > CASSANDRA-18228 seems to have deleted [the section of documentation about > dynamic data masking > (CEP-20)|https://github.com/apache/cassandra/blob/63ab8e09286e4b2076695f62e08a7f39cb6860d1/doc/modules/cassandra/pages/cql/dynamic_data_masking.adoc]. > The code examples and the (broken) index entry are still present. This > ticket should add it back. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18776) Restore accidentally deleted documentation for DDM
Andres de la Peña created CASSANDRA-18776: - Summary: Restore accidentally deleted documentation for DDM Key: CASSANDRA-18776 URL: https://issues.apache.org/jira/browse/CASSANDRA-18776 Project: Cassandra Issue Type: Bug Components: Documentation Reporter: Andres de la Peña Assignee: Andres de la Peña CASSANDRA-18228 seems to have deleted [the section of documentation about dynamic data masking (CEP-20)|https://github.com/apache/cassandra/blob/63ab8e09286e4b2076695f62e08a7f39cb6860d1/doc/modules/cassandra/pages/cql/dynamic_data_masking.adoc]. The code examples and the (broken) index entry are still present. This ticket should add it back. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18743) Remove dependency on archived project metrics-reporter-config
[ https://issues.apache.org/jira/browse/CASSANDRA-18743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18743: --- Fix Version/s: 5.0.x (was: 5.0) > Remove dependency on archived project metrics-reporter-config > - > > Key: CASSANDRA-18743 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18743 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Abe Ratnofsky >Assignee: Maxim Muzafarov >Priority: Normal > Fix For: 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > metrics-reporter-config has been archived for over 3 years, and has received > no updates in nearly 6 years: > https://github.com/addthis/metrics-reporter-config > metrics-reporter-config processes YAML unsafely: > https://github.com/addthis/metrics-reporter-config/blob/master/reporter-config-base/src/main/java/com/addthis/metrics/reporter/config/AbstractReporterConfig.java#L39C34-L39C45 > This is related to CASSANDRA-18614, which is focused on migrating to safe > YAML processing for the main configuration files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18772) Remove dependency on commons-codec
[ https://issues.apache.org/jira/browse/CASSANDRA-18772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755642#comment-17755642 ] Ekaterina Dimitrova commented on CASSANDRA-18772: - ML thread started here - https://lists.apache.org/thread/9m1vz5qyows97wlppkwk1fd8386rj9q1 > Remove dependency on commons-codec > -- > > Key: CASSANDRA-18772 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18772 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 5.0.x, 5.x > > > The dependency doesn't seem to have any direct usage in our codebase. > The only reference to how it might be used was found in CASSANDRA-12790. > Considering we also deprecate in 5.0 and remove metrics-reporter-config as > part of CASSANDRA-18743, I do not see a reason not to remove this one too. > We should probably take the same approach as CASSANDRA-18743 - deprecate in > 5.0 and remove in 5.1. ML thread is to be opened too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18161) Test Failure: org.apache.cassandra.transport.CQLConnectionTest.handleCorruptionOfLargeMessageFrame
[
https://issues.apache.org/jira/browse/CASSANDRA-18161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755640#comment-17755640
]
Caleb Rackliffe commented on CASSANDRA-18161:
-
Observed this in a recent run against a trunk-based branch:
[https://app.circleci.com/pipelines/github/adelapena/cassandra/3104/workflows/7d5e837f-8db3-4888-bd67-6fadf8511ca3/jobs/67840/tests#failed-test-0]
> Test Failure:
> org.apache.cassandra.transport.CQLConnectionTest.handleCorruptionOfLargeMessageFrame
> --
>
> Key: CASSANDRA-18161
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18161
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
>
> Flaky.
> {noformat}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.transport.CQLConnectionTest.testFrameCorruption(CQLConnectionTest.java:491)
> at
> org.apache.cassandra.transport.CQLConnectionTest.testFrameCorruption(CQLConnectionTest.java:450)
> at
> org.apache.cassandra.transport.CQLConnectionTest.handleCorruptionOfLargeMessageFrame(CQLConnectionTest.java:221)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18673) Reduce size of per-SSTable index components
[ https://issues.apache.org/jira/browse/CASSANDRA-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18673: Fix Version/s: 5.0 5.1 (was: 5.x) (was: 5.0.x) > Reduce size of per-SSTable index components > --- > > Key: CASSANDRA-18673 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18673 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0, 5.1 > > Time Spent: 6.5h > Remaining Estimate: 0h > > The current per-SSTable index components are large because the primary keys > that are stored in them include the token as part of the byte comparable. The > byte comparable puts the token first meaning that we get very little prefix > compression from either the trie or the sorted terms store. > We can fix this by removing the token from the primary key serialization. > This would allow us to get the prefix compression from the trie and the > sorted terms store. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18673) Reduce size of per-SSTable index components
[ https://issues.apache.org/jira/browse/CASSANDRA-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755633#comment-17755633 ] Caleb Rackliffe edited comment on CASSANDRA-18673 at 8/17/23 4:58 PM: -- 5.0 and trunk squashed and rebased here, thanks to [~adelapena]: |[5.0|https://github.com/adelapena/cassandra/commits/18673-5.0-review] |[CircleCI|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=18673-5.0-review]| |[trunk|https://github.com/adelapena/cassandra/commits/18673-trunk-review] |[CircleCI|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=18673-trunk-review]| was (Author: maedhroz): 5.0 and trunk squashed and rebased here, thanks to [~adelapena]: [5.0|https://github.com/adelapena/cassandra/commits/18673-5.0-review] [CircleCI|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=18673-5.0-review] [trunk|https://github.com/adelapena/cassandra/commits/18673-trunk-review] [CircleCI|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=18673-trunk-review] > Reduce size of per-SSTable index components > --- > > Key: CASSANDRA-18673 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18673 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0.x, 5.x > > Time Spent: 6.5h > Remaining Estimate: 0h > > The current per-SSTable index components are large because the primary keys > that are stored in them include the token as part of the byte comparable. The > byte comparable puts the token first meaning that we get very little prefix > compression from either the trie or the sorted terms store. > We can fix this by removing the token from the primary key serialization. > This would allow us to get the prefix compression from the trie and the > sorted terms store. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18673) Reduce size of per-SSTable index components
[ https://issues.apache.org/jira/browse/CASSANDRA-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18673: Status: Ready to Commit (was: Review In Progress) 5.0 and trunk squashed and rebased here, thanks to [~adelapena]: [5.0|https://github.com/adelapena/cassandra/commits/18673-5.0-review] [CircleCI|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=18673-5.0-review] [trunk|https://github.com/adelapena/cassandra/commits/18673-trunk-review] [CircleCI|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=18673-trunk-review] > Reduce size of per-SSTable index components > --- > > Key: CASSANDRA-18673 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18673 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0.x, 5.x > > Time Spent: 6.5h > Remaining Estimate: 0h > > The current per-SSTable index components are large because the primary keys > that are stored in them include the token as part of the byte comparable. The > byte comparable puts the token first meaning that we get very little prefix > compression from either the trie or the sorted terms store. > We can fix this by removing the token from the primary key serialization. > This would allow us to get the prefix compression from the trie and the > sorted terms store. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18775) Remove libraries in lib/sigar-bin for unsupported architectures
[
https://issues.apache.org/jira/browse/CASSANDRA-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755618#comment-17755618
]
Maxim Muzafarov edited comment on CASSANDRA-18775 at 8/17/23 4:45 PM:
--
I'll take a look at it, sure. I think I need a few weeks to polish
CASSANDRA-16360 and will try to figure out how to handle this one.
was (Author: mmuzaf):
I'll take a look at it, sure. I think I need a few weeks to polish
CASSANDRA-16360 and try to figure out how to handle this one.
> Remove libraries in lib/sigar-bin for unsupported architectures
> ---
>
> Key: CASSANDRA-18775
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18775
> Project: Cassandra
> Issue Type: Task
>Reporter: Stefan Miklosovic
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.x
>
>
> {code}
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:41 $ ls -la lib/sigar-bin/
> total 6376
> drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
> drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
> -rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26 libsigar-amd64-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 246605 aug 17 11:26 libsigar-amd64-linux.so
> -rw-rw-r-- 1 fermat fermat 251360 aug 17 11:26 libsigar-amd64-solaris.so
> -rw-rw-r-- 1 fermat fermat 577452 aug 17 11:26 libsigar-ia64-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 494929 aug 17 11:26 libsigar-ia64-linux.so
> -rw-rw-r-- 1 fermat fermat 516096 aug 17 11:26 libsigar-pa-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 425077 aug 17 11:26 libsigar-ppc64-aix-5.so
> -rw-rw-r-- 1 fermat fermat 310792 aug 17 11:26 libsigar-ppc64le-linux.so
> -rw-rw-r-- 1 fermat fermat 330767 aug 17 11:26 libsigar-ppc64-linux.so
> -rw-rw-r-- 1 fermat fermat 400925 aug 17 11:26 libsigar-ppc-aix-5.so
> -rw-rw-r-- 1 fermat fermat 258547 aug 17 11:26 libsigar-ppc-linux.so
> -rw-rw-r-- 1 fermat fermat 269932 aug 17 11:26 libsigar-s390x-linux.so
> -rw-rw-r-- 1 fermat fermat 261896 aug 17 11:26 libsigar-sparc64-solaris.so
> -rw-rw-r-- 1 fermat fermat 285004 aug 17 11:26 libsigar-sparc-solaris.so
> -rw-rw-r-- 1 fermat fermat 397440 aug 17 11:26
> libsigar-universal64-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 377668 aug 17 11:26 libsigar-universal-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 179751 aug 17 11:26 libsigar-x86-freebsd-5.so
> -rw-rw-r-- 1 fermat fermat 179379 aug 17 11:26 libsigar-x86-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 233385 aug 17 11:26 libsigar-x86-linux.so
> -rw-rw-r-- 1 fermat fermat 242880 aug 17 11:26 libsigar-x86-solaris.so
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:43 $ du -shc lib/sigar-bin/
> 6,3Mlib/sigar-bin/
> 6,3Mtotal
> {code}
> I think we could definitely improve this. We basically need just x86-linux,
> amd64-linux and two libs for macosx.
> We could save like 5M from the final tarball size. From 75M down to 70M is
> not bad!
> Or maybe I am not getting something and we need this?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755619#comment-17755619
]
Brandon Williams commented on CASSANDRA-18710:
--
Now I suspect the problem is adding the flush fixed the instance where we need
to flush, but also caused a problem when we don't, making us flush a small
value.
> Test failure:
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from
> org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
> --
>
> Key: CASSANDRA-18710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18710
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.x
>
>
> Seen here:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1644/testReport/org.apache.cassandra.io/DiskSpaceMetricsTest/testFlushSize__jdk17/]
> h3.
> {code:java}
> Error Message
> expected:<7200.0> but was:<1367.83970468544>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<7200.0> but
> was:<1367.83970468544> at
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize(DiskSpaceMetricsTest.java:119)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18775) Remove libraries in lib/sigar-bin for unsupported architectures
[
https://issues.apache.org/jira/browse/CASSANDRA-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755618#comment-17755618
]
Maxim Muzafarov edited comment on CASSANDRA-18775 at 8/17/23 4:28 PM:
--
I'll take a look at it, sure. I think I need a few weeks to polish
CASSANDRA-16360 and try to figure out how to handle this one.
was (Author: mmuzaf):
[~mmuzaf] I'll take a look at it, sure. I think I need a few weeks to polish
CASSANDRA-16360 and try to figure out how to handle this one.
> Remove libraries in lib/sigar-bin for unsupported architectures
> ---
>
> Key: CASSANDRA-18775
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18775
> Project: Cassandra
> Issue Type: Task
>Reporter: Stefan Miklosovic
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.x
>
>
> {code}
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:41 $ ls -la lib/sigar-bin/
> total 6376
> drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
> drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
> -rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26 libsigar-amd64-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 246605 aug 17 11:26 libsigar-amd64-linux.so
> -rw-rw-r-- 1 fermat fermat 251360 aug 17 11:26 libsigar-amd64-solaris.so
> -rw-rw-r-- 1 fermat fermat 577452 aug 17 11:26 libsigar-ia64-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 494929 aug 17 11:26 libsigar-ia64-linux.so
> -rw-rw-r-- 1 fermat fermat 516096 aug 17 11:26 libsigar-pa-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 425077 aug 17 11:26 libsigar-ppc64-aix-5.so
> -rw-rw-r-- 1 fermat fermat 310792 aug 17 11:26 libsigar-ppc64le-linux.so
> -rw-rw-r-- 1 fermat fermat 330767 aug 17 11:26 libsigar-ppc64-linux.so
> -rw-rw-r-- 1 fermat fermat 400925 aug 17 11:26 libsigar-ppc-aix-5.so
> -rw-rw-r-- 1 fermat fermat 258547 aug 17 11:26 libsigar-ppc-linux.so
> -rw-rw-r-- 1 fermat fermat 269932 aug 17 11:26 libsigar-s390x-linux.so
> -rw-rw-r-- 1 fermat fermat 261896 aug 17 11:26 libsigar-sparc64-solaris.so
> -rw-rw-r-- 1 fermat fermat 285004 aug 17 11:26 libsigar-sparc-solaris.so
> -rw-rw-r-- 1 fermat fermat 397440 aug 17 11:26
> libsigar-universal64-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 377668 aug 17 11:26 libsigar-universal-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 179751 aug 17 11:26 libsigar-x86-freebsd-5.so
> -rw-rw-r-- 1 fermat fermat 179379 aug 17 11:26 libsigar-x86-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 233385 aug 17 11:26 libsigar-x86-linux.so
> -rw-rw-r-- 1 fermat fermat 242880 aug 17 11:26 libsigar-x86-solaris.so
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:43 $ du -shc lib/sigar-bin/
> 6,3Mlib/sigar-bin/
> 6,3Mtotal
> {code}
> I think we could definitely improve this. We basically need just x86-linux,
> amd64-linux and two libs for macosx.
> We could save like 5M from the final tarball size. From 75M down to 70M is
> not bad!
> Or maybe I am not getting something and we need this?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18775) Remove libraries in lib/sigar-bin for unsupported architectures
[
https://issues.apache.org/jira/browse/CASSANDRA-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755618#comment-17755618
]
Maxim Muzafarov commented on CASSANDRA-18775:
-
[~mmuzaf] I'll take a look at it, sure. I think I need a few weeks to polish
CASSANDRA-16360 and try to figure out how to handle this one.
> Remove libraries in lib/sigar-bin for unsupported architectures
> ---
>
> Key: CASSANDRA-18775
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18775
> Project: Cassandra
> Issue Type: Task
>Reporter: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
>
> {code}
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:41 $ ls -la lib/sigar-bin/
> total 6376
> drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
> drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
> -rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26 libsigar-amd64-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 246605 aug 17 11:26 libsigar-amd64-linux.so
> -rw-rw-r-- 1 fermat fermat 251360 aug 17 11:26 libsigar-amd64-solaris.so
> -rw-rw-r-- 1 fermat fermat 577452 aug 17 11:26 libsigar-ia64-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 494929 aug 17 11:26 libsigar-ia64-linux.so
> -rw-rw-r-- 1 fermat fermat 516096 aug 17 11:26 libsigar-pa-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 425077 aug 17 11:26 libsigar-ppc64-aix-5.so
> -rw-rw-r-- 1 fermat fermat 310792 aug 17 11:26 libsigar-ppc64le-linux.so
> -rw-rw-r-- 1 fermat fermat 330767 aug 17 11:26 libsigar-ppc64-linux.so
> -rw-rw-r-- 1 fermat fermat 400925 aug 17 11:26 libsigar-ppc-aix-5.so
> -rw-rw-r-- 1 fermat fermat 258547 aug 17 11:26 libsigar-ppc-linux.so
> -rw-rw-r-- 1 fermat fermat 269932 aug 17 11:26 libsigar-s390x-linux.so
> -rw-rw-r-- 1 fermat fermat 261896 aug 17 11:26 libsigar-sparc64-solaris.so
> -rw-rw-r-- 1 fermat fermat 285004 aug 17 11:26 libsigar-sparc-solaris.so
> -rw-rw-r-- 1 fermat fermat 397440 aug 17 11:26
> libsigar-universal64-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 377668 aug 17 11:26 libsigar-universal-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 179751 aug 17 11:26 libsigar-x86-freebsd-5.so
> -rw-rw-r-- 1 fermat fermat 179379 aug 17 11:26 libsigar-x86-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 233385 aug 17 11:26 libsigar-x86-linux.so
> -rw-rw-r-- 1 fermat fermat 242880 aug 17 11:26 libsigar-x86-solaris.so
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:43 $ du -shc lib/sigar-bin/
> 6,3Mlib/sigar-bin/
> 6,3Mtotal
> {code}
> I think we could definitely improve this. We basically need just x86-linux,
> amd64-linux and two libs for macosx.
> We could save like 5M from the final tarball size. From 75M down to 70M is
> not bad!
> Or maybe I am not getting something and we need this?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18775) Remove libraries in lib/sigar-bin for unsupported architectures
[
https://issues.apache.org/jira/browse/CASSANDRA-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov reassigned CASSANDRA-18775:
---
Assignee: Maxim Muzafarov
> Remove libraries in lib/sigar-bin for unsupported architectures
> ---
>
> Key: CASSANDRA-18775
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18775
> Project: Cassandra
> Issue Type: Task
>Reporter: Stefan Miklosovic
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.x
>
>
> {code}
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:41 $ ls -la lib/sigar-bin/
> total 6376
> drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
> drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
> -rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26 libsigar-amd64-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 246605 aug 17 11:26 libsigar-amd64-linux.so
> -rw-rw-r-- 1 fermat fermat 251360 aug 17 11:26 libsigar-amd64-solaris.so
> -rw-rw-r-- 1 fermat fermat 577452 aug 17 11:26 libsigar-ia64-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 494929 aug 17 11:26 libsigar-ia64-linux.so
> -rw-rw-r-- 1 fermat fermat 516096 aug 17 11:26 libsigar-pa-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 425077 aug 17 11:26 libsigar-ppc64-aix-5.so
> -rw-rw-r-- 1 fermat fermat 310792 aug 17 11:26 libsigar-ppc64le-linux.so
> -rw-rw-r-- 1 fermat fermat 330767 aug 17 11:26 libsigar-ppc64-linux.so
> -rw-rw-r-- 1 fermat fermat 400925 aug 17 11:26 libsigar-ppc-aix-5.so
> -rw-rw-r-- 1 fermat fermat 258547 aug 17 11:26 libsigar-ppc-linux.so
> -rw-rw-r-- 1 fermat fermat 269932 aug 17 11:26 libsigar-s390x-linux.so
> -rw-rw-r-- 1 fermat fermat 261896 aug 17 11:26 libsigar-sparc64-solaris.so
> -rw-rw-r-- 1 fermat fermat 285004 aug 17 11:26 libsigar-sparc-solaris.so
> -rw-rw-r-- 1 fermat fermat 397440 aug 17 11:26
> libsigar-universal64-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 377668 aug 17 11:26 libsigar-universal-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 179751 aug 17 11:26 libsigar-x86-freebsd-5.so
> -rw-rw-r-- 1 fermat fermat 179379 aug 17 11:26 libsigar-x86-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 233385 aug 17 11:26 libsigar-x86-linux.so
> -rw-rw-r-- 1 fermat fermat 242880 aug 17 11:26 libsigar-x86-solaris.so
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:43 $ du -shc lib/sigar-bin/
> 6,3Mlib/sigar-bin/
> 6,3Mtotal
> {code}
> I think we could definitely improve this. We basically need just x86-linux,
> amd64-linux and two libs for macosx.
> We could save like 5M from the final tarball size. From 75M down to 70M is
> not bad!
> Or maybe I am not getting something and we need this?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18775) Remove libraries in lib/sigar-bin for unsupported architectures
[
https://issues.apache.org/jira/browse/CASSANDRA-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755604#comment-17755604
]
Stefan Miklosovic commented on CASSANDRA-18775:
---
I think that [~mmuzaf] is going to take a look at this as he said that the
similar stuff was in Ignite, fingers crossed we move this somehow forward!
> Remove libraries in lib/sigar-bin for unsupported architectures
> ---
>
> Key: CASSANDRA-18775
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18775
> Project: Cassandra
> Issue Type: Task
>Reporter: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
>
> {code}
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:41 $ ls -la lib/sigar-bin/
> total 6376
> drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
> drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
> -rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26 libsigar-amd64-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 246605 aug 17 11:26 libsigar-amd64-linux.so
> -rw-rw-r-- 1 fermat fermat 251360 aug 17 11:26 libsigar-amd64-solaris.so
> -rw-rw-r-- 1 fermat fermat 577452 aug 17 11:26 libsigar-ia64-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 494929 aug 17 11:26 libsigar-ia64-linux.so
> -rw-rw-r-- 1 fermat fermat 516096 aug 17 11:26 libsigar-pa-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 425077 aug 17 11:26 libsigar-ppc64-aix-5.so
> -rw-rw-r-- 1 fermat fermat 310792 aug 17 11:26 libsigar-ppc64le-linux.so
> -rw-rw-r-- 1 fermat fermat 330767 aug 17 11:26 libsigar-ppc64-linux.so
> -rw-rw-r-- 1 fermat fermat 400925 aug 17 11:26 libsigar-ppc-aix-5.so
> -rw-rw-r-- 1 fermat fermat 258547 aug 17 11:26 libsigar-ppc-linux.so
> -rw-rw-r-- 1 fermat fermat 269932 aug 17 11:26 libsigar-s390x-linux.so
> -rw-rw-r-- 1 fermat fermat 261896 aug 17 11:26 libsigar-sparc64-solaris.so
> -rw-rw-r-- 1 fermat fermat 285004 aug 17 11:26 libsigar-sparc-solaris.so
> -rw-rw-r-- 1 fermat fermat 397440 aug 17 11:26
> libsigar-universal64-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 377668 aug 17 11:26 libsigar-universal-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 179751 aug 17 11:26 libsigar-x86-freebsd-5.so
> -rw-rw-r-- 1 fermat fermat 179379 aug 17 11:26 libsigar-x86-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 233385 aug 17 11:26 libsigar-x86-linux.so
> -rw-rw-r-- 1 fermat fermat 242880 aug 17 11:26 libsigar-x86-solaris.so
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:43 $ du -shc lib/sigar-bin/
> 6,3Mlib/sigar-bin/
> 6,3Mtotal
> {code}
> I think we could definitely improve this. We basically need just x86-linux,
> amd64-linux and two libs for macosx.
> We could save like 5M from the final tarball size. From 75M down to 70M is
> not bad!
> Or maybe I am not getting something and we need this?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18534) Make sstable format configurable per table
[
https://issues.apache.org/jira/browse/CASSANDRA-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755603#comment-17755603
]
Maxwell Guo commented on CASSANDRA-18534:
-
ok,thanks
> Make sstable format configurable per table
> --
>
> Key: CASSANDRA-18534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18534
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
>
> Some SSTable format settings need to be configurable per table for better
> efficiency. This includes:
> - {{row_index_granularity}}
> - {{bloom_filter_fp_chance}}
> - {{crc_check_chance}}
> - {{min/max_index_interval}}
> Some of these are currently configurable using direct properties of tables.
> Having them as format properties makes better sense and should also support
> specifying useable combinations of settings, e.g.
> {code:java}
> CREATE TABLE ... WITH sstable_format = "bti-fast";
> CREATE TABLE ... WITH sstable_format = "bti-small";
> {code}
> where {{bti-fast}} and {{bti-small}} can be defined in {{cassandra.yaml}}
> e.g. as
> {code:java}
> sstable.format.options:
> - bti-fast:
> row_index_granularity: 1kiB
> bloom_filter_fp_chance: 0.01
> - bti-small:
> row_index_granularity: 32kiB
> bloom_filter_fp_chance: 0.1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18760) Need to block the ability to DROP table column of 'map' type and add back as 'blob' type
[
https://issues.apache.org/jira/browse/CASSANDRA-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755601#comment-17755601
]
Andres de la Peña commented on CASSANDRA-18760:
---
Looks good to me, let's see how CI goes.
> Need to block the ability to DROP table column of 'map' type and add back as
> 'blob' type
>
>
> Key: CASSANDRA-18760
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18760
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Paul Ayers
>Assignee: Stefan Miklosovic
>Priority: Urgent
> Labels: 4.0.10
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Recently hit an un-recoverable situation in Cassandra 4.0.10 after dropping a
> 'map' column and adding it back as a 'blob', which caused corruption that
> neither offline nor online scrub could fix.
> When dropping a 'blob' column and attempting to add it back as a 'map' type,
> the operation is blocked with:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> re-add previously dropped column 'col1' of type map,
> incompatible with previous type blob"
> {code}
> We need to do the same going from 'map' to 'blob' to avoid this potentially
> very bad scenario that can cause data loss.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18775) Remove libraries in lib/sigar-bin for unsupported architectures
[
https://issues.apache.org/jira/browse/CASSANDRA-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755600#comment-17755600
]
Brandon Williams commented on CASSANDRA-18775:
--
I think the only reason to have it was windows, which is no longer supported.
> Remove libraries in lib/sigar-bin for unsupported architectures
> ---
>
> Key: CASSANDRA-18775
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18775
> Project: Cassandra
> Issue Type: Task
>Reporter: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
>
> {code}
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:41 $ ls -la lib/sigar-bin/
> total 6376
> drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
> drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
> -rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26 libsigar-amd64-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 246605 aug 17 11:26 libsigar-amd64-linux.so
> -rw-rw-r-- 1 fermat fermat 251360 aug 17 11:26 libsigar-amd64-solaris.so
> -rw-rw-r-- 1 fermat fermat 577452 aug 17 11:26 libsigar-ia64-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 494929 aug 17 11:26 libsigar-ia64-linux.so
> -rw-rw-r-- 1 fermat fermat 516096 aug 17 11:26 libsigar-pa-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 425077 aug 17 11:26 libsigar-ppc64-aix-5.so
> -rw-rw-r-- 1 fermat fermat 310792 aug 17 11:26 libsigar-ppc64le-linux.so
> -rw-rw-r-- 1 fermat fermat 330767 aug 17 11:26 libsigar-ppc64-linux.so
> -rw-rw-r-- 1 fermat fermat 400925 aug 17 11:26 libsigar-ppc-aix-5.so
> -rw-rw-r-- 1 fermat fermat 258547 aug 17 11:26 libsigar-ppc-linux.so
> -rw-rw-r-- 1 fermat fermat 269932 aug 17 11:26 libsigar-s390x-linux.so
> -rw-rw-r-- 1 fermat fermat 261896 aug 17 11:26 libsigar-sparc64-solaris.so
> -rw-rw-r-- 1 fermat fermat 285004 aug 17 11:26 libsigar-sparc-solaris.so
> -rw-rw-r-- 1 fermat fermat 397440 aug 17 11:26
> libsigar-universal64-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 377668 aug 17 11:26 libsigar-universal-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 179751 aug 17 11:26 libsigar-x86-freebsd-5.so
> -rw-rw-r-- 1 fermat fermat 179379 aug 17 11:26 libsigar-x86-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 233385 aug 17 11:26 libsigar-x86-linux.so
> -rw-rw-r-- 1 fermat fermat 242880 aug 17 11:26 libsigar-x86-solaris.so
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:43 $ du -shc lib/sigar-bin/
> 6,3Mlib/sigar-bin/
> 6,3Mtotal
> {code}
> I think we could definitely improve this. We basically need just x86-linux,
> amd64-linux and two libs for macosx.
> We could save like 5M from the final tarball size. From 75M down to 70M is
> not bad!
> Or maybe I am not getting something and we need this?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18775) Remove libraries in lib/sigar-bin for unsupported architectures
[
https://issues.apache.org/jira/browse/CASSANDRA-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755599#comment-17755599
]
Stefan Miklosovic commented on CASSANDRA-18775:
---
Thanks Brandon, it would be great if we found a way how to get rid of this
completely.
> Remove libraries in lib/sigar-bin for unsupported architectures
> ---
>
> Key: CASSANDRA-18775
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18775
> Project: Cassandra
> Issue Type: Task
>Reporter: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
>
> {code}
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:41 $ ls -la lib/sigar-bin/
> total 6376
> drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
> drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
> -rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26 libsigar-amd64-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 246605 aug 17 11:26 libsigar-amd64-linux.so
> -rw-rw-r-- 1 fermat fermat 251360 aug 17 11:26 libsigar-amd64-solaris.so
> -rw-rw-r-- 1 fermat fermat 577452 aug 17 11:26 libsigar-ia64-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 494929 aug 17 11:26 libsigar-ia64-linux.so
> -rw-rw-r-- 1 fermat fermat 516096 aug 17 11:26 libsigar-pa-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 425077 aug 17 11:26 libsigar-ppc64-aix-5.so
> -rw-rw-r-- 1 fermat fermat 310792 aug 17 11:26 libsigar-ppc64le-linux.so
> -rw-rw-r-- 1 fermat fermat 330767 aug 17 11:26 libsigar-ppc64-linux.so
> -rw-rw-r-- 1 fermat fermat 400925 aug 17 11:26 libsigar-ppc-aix-5.so
> -rw-rw-r-- 1 fermat fermat 258547 aug 17 11:26 libsigar-ppc-linux.so
> -rw-rw-r-- 1 fermat fermat 269932 aug 17 11:26 libsigar-s390x-linux.so
> -rw-rw-r-- 1 fermat fermat 261896 aug 17 11:26 libsigar-sparc64-solaris.so
> -rw-rw-r-- 1 fermat fermat 285004 aug 17 11:26 libsigar-sparc-solaris.so
> -rw-rw-r-- 1 fermat fermat 397440 aug 17 11:26
> libsigar-universal64-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 377668 aug 17 11:26 libsigar-universal-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 179751 aug 17 11:26 libsigar-x86-freebsd-5.so
> -rw-rw-r-- 1 fermat fermat 179379 aug 17 11:26 libsigar-x86-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 233385 aug 17 11:26 libsigar-x86-linux.so
> -rw-rw-r-- 1 fermat fermat 242880 aug 17 11:26 libsigar-x86-solaris.so
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:43 $ du -shc lib/sigar-bin/
> 6,3Mlib/sigar-bin/
> 6,3Mtotal
> {code}
> I think we could definitely improve this. We basically need just x86-linux,
> amd64-linux and two libs for macosx.
> We could save like 5M from the final tarball size. From 75M down to 70M is
> not bad!
> Or maybe I am not getting something and we need this?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18534) Make sstable format configurable per table
[
https://issues.apache.org/jira/browse/CASSANDRA-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755595#comment-17755595
]
Branimir Lambov commented on CASSANDRA-18534:
-
No such validation is needed. When we receive an {{ALTER/CREATE}} statement we
validate that it starts with a known format name, that the specific variation
is specified in the yaml, and that the definition in the yaml is
understood/validated by the format.
> Make sstable format configurable per table
> --
>
> Key: CASSANDRA-18534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18534
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
>
> Some SSTable format settings need to be configurable per table for better
> efficiency. This includes:
> - {{row_index_granularity}}
> - {{bloom_filter_fp_chance}}
> - {{crc_check_chance}}
> - {{min/max_index_interval}}
> Some of these are currently configurable using direct properties of tables.
> Having them as format properties makes better sense and should also support
> specifying useable combinations of settings, e.g.
> {code:java}
> CREATE TABLE ... WITH sstable_format = "bti-fast";
> CREATE TABLE ... WITH sstable_format = "bti-small";
> {code}
> where {{bti-fast}} and {{bti-small}} can be defined in {{cassandra.yaml}}
> e.g. as
> {code:java}
> sstable.format.options:
> - bti-fast:
> row_index_granularity: 1kiB
> bloom_filter_fp_chance: 0.01
> - bti-small:
> row_index_granularity: 32kiB
> bloom_filter_fp_chance: 0.1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18534) Make sstable format configurable per table
[
https://issues.apache.org/jira/browse/CASSANDRA-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755591#comment-17755591
]
Maxwell Guo commented on CASSANDRA-18534:
-
So that means there is no need for me to do any validation for "variation" such
as "bit-fast"'s fast ? right ? [~blambov]
> Make sstable format configurable per table
> --
>
> Key: CASSANDRA-18534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18534
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
>
> Some SSTable format settings need to be configurable per table for better
> efficiency. This includes:
> - {{row_index_granularity}}
> - {{bloom_filter_fp_chance}}
> - {{crc_check_chance}}
> - {{min/max_index_interval}}
> Some of these are currently configurable using direct properties of tables.
> Having them as format properties makes better sense and should also support
> specifying useable combinations of settings, e.g.
> {code:java}
> CREATE TABLE ... WITH sstable_format = "bti-fast";
> CREATE TABLE ... WITH sstable_format = "bti-small";
> {code}
> where {{bti-fast}} and {{bti-small}} can be defined in {{cassandra.yaml}}
> e.g. as
> {code:java}
> sstable.format.options:
> - bti-fast:
> row_index_granularity: 1kiB
> bloom_filter_fp_chance: 0.01
> - bti-small:
> row_index_granularity: 32kiB
> bloom_filter_fp_chance: 0.1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18767) tablestats should show speculative retries
[ https://issues.apache.org/jira/browse/CASSANDRA-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxwell Guo updated CASSANDRA-18767: Status: Needs Committer (was: Review In Progress) > tablestats should show speculative retries > -- > > Key: CASSANDRA-18767 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18767 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > NodeProbe makes > [provisions|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java#L1921] > for it, but it is never used later. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18775) Remove libraries in lib/sigar-bin for unsupported architectures
[
https://issues.apache.org/jira/browse/CASSANDRA-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755583#comment-17755583
]
Brandon Williams commented on CASSANDRA-18775:
--
Maybe a dupe of CASSANDRA-16565
> Remove libraries in lib/sigar-bin for unsupported architectures
> ---
>
> Key: CASSANDRA-18775
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18775
> Project: Cassandra
> Issue Type: Task
>Reporter: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
>
> {code}
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:41 $ ls -la lib/sigar-bin/
> total 6376
> drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
> drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
> -rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26 libsigar-amd64-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 246605 aug 17 11:26 libsigar-amd64-linux.so
> -rw-rw-r-- 1 fermat fermat 251360 aug 17 11:26 libsigar-amd64-solaris.so
> -rw-rw-r-- 1 fermat fermat 577452 aug 17 11:26 libsigar-ia64-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 494929 aug 17 11:26 libsigar-ia64-linux.so
> -rw-rw-r-- 1 fermat fermat 516096 aug 17 11:26 libsigar-pa-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 425077 aug 17 11:26 libsigar-ppc64-aix-5.so
> -rw-rw-r-- 1 fermat fermat 310792 aug 17 11:26 libsigar-ppc64le-linux.so
> -rw-rw-r-- 1 fermat fermat 330767 aug 17 11:26 libsigar-ppc64-linux.so
> -rw-rw-r-- 1 fermat fermat 400925 aug 17 11:26 libsigar-ppc-aix-5.so
> -rw-rw-r-- 1 fermat fermat 258547 aug 17 11:26 libsigar-ppc-linux.so
> -rw-rw-r-- 1 fermat fermat 269932 aug 17 11:26 libsigar-s390x-linux.so
> -rw-rw-r-- 1 fermat fermat 261896 aug 17 11:26 libsigar-sparc64-solaris.so
> -rw-rw-r-- 1 fermat fermat 285004 aug 17 11:26 libsigar-sparc-solaris.so
> -rw-rw-r-- 1 fermat fermat 397440 aug 17 11:26
> libsigar-universal64-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 377668 aug 17 11:26 libsigar-universal-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 179751 aug 17 11:26 libsigar-x86-freebsd-5.so
> -rw-rw-r-- 1 fermat fermat 179379 aug 17 11:26 libsigar-x86-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 233385 aug 17 11:26 libsigar-x86-linux.so
> -rw-rw-r-- 1 fermat fermat 242880 aug 17 11:26 libsigar-x86-solaris.so
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:43 $ du -shc lib/sigar-bin/
> 6,3Mlib/sigar-bin/
> 6,3Mtotal
> {code}
> I think we could definitely improve this. We basically need just x86-linux,
> amd64-linux and two libs for macosx.
> We could save like 5M from the final tarball size. From 75M down to 70M is
> not bad!
> Or maybe I am not getting something and we need this?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755582#comment-17755582 ] Michael Semb Wever commented on CASSANDRA-18688: What are users asking for? JDKs in between 11 or 17 or only newer above 17 ? How difficult is it to edit the bin/cassandra script (or cassandra.in.sh) for them to get what they want ? What signal (advertently or not) are we giving by providing the `-J` option ? Would a env var be a better signal ? (I think the use of `-J`, if we accept it, should still print a loud warning.) [~skokoori], I'm interested in your opinion here, as you are best representative of the user here. How does this impact the other scripts ? I know [~claude] was digging into the .in.sh scripts, as they are a bit of a mess. I agree that the scripts should by default fail-fast if a jdk not included in `java.supported` is used. > Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out > of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.0.x > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18760) Need to block the ability to DROP table column of 'map' type and add back as 'blob' type
[
https://issues.apache.org/jira/browse/CASSANDRA-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Ayers updated CASSANDRA-18760:
---
Description:
Recently hit an un-recoverable situation in Cassandra 4.0.10 after dropping a
'map' column and adding it back as a 'blob', which caused corruption that
neither offline nor online scrub could fix.
When dropping a 'blob' column and attempting to add it back as a 'map' type,
the operation is blocked with:
{code:java}
InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
re-add previously dropped column 'col1' of type map, incompatible
with previous type blob"
{code}
We need to do the same going from 'map' to 'blob' to avoid this potentially
very bad scenario that can cause data loss.
was:
Recently hit an uncoverable situation in Cassandra 4.0.10 after dropping a
'map' column and adding it back as a 'blob', which caused corruption that
neither offline nor online scrub could fix.
When dropping a 'blob' column and attempting to add it back as a 'map' type,
the operation is blocked with:
{code:java}
InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
re-add previously dropped column 'col1' of type map, incompatible
with previous type blob"
{code}
We need to do the same going from 'map' to 'blob' to avoid this potentially
very bad scenario that can cause data loss.
> Need to block the ability to DROP table column of 'map' type and add back as
> 'blob' type
>
>
> Key: CASSANDRA-18760
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18760
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Paul Ayers
>Assignee: Stefan Miklosovic
>Priority: Urgent
> Labels: 4.0.10
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Recently hit an un-recoverable situation in Cassandra 4.0.10 after dropping a
> 'map' column and adding it back as a 'blob', which caused corruption that
> neither offline nor online scrub could fix.
> When dropping a 'blob' column and attempting to add it back as a 'map' type,
> the operation is blocked with:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> re-add previously dropped column 'col1' of type map,
> incompatible with previous type blob"
> {code}
> We need to do the same going from 'map' to 'blob' to avoid this potentially
> very bad scenario that can cause data loss.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18176) Merged SSTable files not reclaimed by OS
[
https://issues.apache.org/jira/browse/CASSANDRA-18176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18176:
-
Resolution: Fixed
Status: Resolved (was: Open)
> Merged SSTable files not reclaimed by OS
>
>
> Key: CASSANDRA-18176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18176
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Pere Balaguer
>Priority: Normal
> Fix For: 4.0.9
>
> Attachments: cassandra_18332-20230415.txt
>
>
> (EDIT: Looks like this is masked by the lack of backport of CASSANDRA-17205
> just on the 4.0 line - will backport that and block here on it)
> After upgrading to Cassandra 4.x (4.0.1 and 4.0.5) we've noticed that at
> times after a compaction deleted sstables diskspace doesn't get reclaimed by
> the OS until the cassandra process is restarted (which kinda points at some
> sort of resource leak), I do not recall this happening in cassandra 3, at
> least not to such degree.
> We've seen the behavior in multiple clusters with different schemas, access
> patterns and consistency levels at somewhat "random" points in time, the only
> interesting thing is that there were active repair sessions at the time
> affecting the node, keyspace and table.
> {noformat}
> $ date +%Y-%m-%d
> 2023-01-17
> $ nodetool version
> ReleaseVersion: 4.0.5
> {noformat}
> {noformat}
> $ lsof +L1 | grep cassandra | grep myawesomecluster | wc -l
> 2772
> $ lsof +L1 | grep cassandra | grep myawesomecluster | tail -n1
> java 59003 cassandra *979u REG 253,810 0 1208053768
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Digest.crc32
> (deleted)
> {noformat}
> {noformat}
> $ grep 2274426 /cassandra/systemlog.log
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,969 BigTableZeroCopyWriter.java:203 - Writing component DATA to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Data.db
> length 2.900KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,969 BigTableZeroCopyWriter.java:213 - Block Writing component to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Data.db
> length 2.900KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,970 BigTableZeroCopyWriter.java:203 - Writing component
> PRIMARY_INDEX to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Index.db
> length 3.739KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,970 BigTableZeroCopyWriter.java:213 - Block Writing component to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Index.db
> length 3.739KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,970 BigTableZeroCopyWriter.java:203 - Writing component STATS to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Statistics.db
> length 5.062KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,970 BigTableZeroCopyWriter.java:213 - Block Writing component to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Statistics.db
> length 5.062KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,971 BigTableZeroCopyWriter.java:203 - Writing component
> COMPRESSION_INFO to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-CompressionInfo.db
> length 0.054KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,971 BigTableZeroCopyWriter.java:213 - Block Writing component to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-CompressionInfo.db
> length 0.054KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,971 BigTableZeroCopyWriter.java:203 - Writing component FILTER to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Filter.db
> length 0.031KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,971 BigTableZeroCopyWriter.java:213 - Block Writing component to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Filter.db
> length 0.031KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,972 BigTableZeroCopyWriter.java:203 - Writing component SUMMARY to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Summary.db
> length 0.436KiB
> INFO
[jira] [Updated] (CASSANDRA-18176) Merged SSTable files not reclaimed by OS
[
https://issues.apache.org/jira/browse/CASSANDRA-18176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18176:
-
Fix Version/s: 4.0.9
(was: 4.0.x)
> Merged SSTable files not reclaimed by OS
>
>
> Key: CASSANDRA-18176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18176
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Pere Balaguer
>Priority: Normal
> Fix For: 4.0.9
>
> Attachments: cassandra_18332-20230415.txt
>
>
> (EDIT: Looks like this is masked by the lack of backport of CASSANDRA-17205
> just on the 4.0 line - will backport that and block here on it)
> After upgrading to Cassandra 4.x (4.0.1 and 4.0.5) we've noticed that at
> times after a compaction deleted sstables diskspace doesn't get reclaimed by
> the OS until the cassandra process is restarted (which kinda points at some
> sort of resource leak), I do not recall this happening in cassandra 3, at
> least not to such degree.
> We've seen the behavior in multiple clusters with different schemas, access
> patterns and consistency levels at somewhat "random" points in time, the only
> interesting thing is that there were active repair sessions at the time
> affecting the node, keyspace and table.
> {noformat}
> $ date +%Y-%m-%d
> 2023-01-17
> $ nodetool version
> ReleaseVersion: 4.0.5
> {noformat}
> {noformat}
> $ lsof +L1 | grep cassandra | grep myawesomecluster | wc -l
> 2772
> $ lsof +L1 | grep cassandra | grep myawesomecluster | tail -n1
> java 59003 cassandra *979u REG 253,810 0 1208053768
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Digest.crc32
> (deleted)
> {noformat}
> {noformat}
> $ grep 2274426 /cassandra/systemlog.log
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,969 BigTableZeroCopyWriter.java:203 - Writing component DATA to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Data.db
> length 2.900KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,969 BigTableZeroCopyWriter.java:213 - Block Writing component to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Data.db
> length 2.900KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,970 BigTableZeroCopyWriter.java:203 - Writing component
> PRIMARY_INDEX to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Index.db
> length 3.739KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,970 BigTableZeroCopyWriter.java:213 - Block Writing component to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Index.db
> length 3.739KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,970 BigTableZeroCopyWriter.java:203 - Writing component STATS to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Statistics.db
> length 5.062KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,970 BigTableZeroCopyWriter.java:213 - Block Writing component to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Statistics.db
> length 5.062KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,971 BigTableZeroCopyWriter.java:203 - Writing component
> COMPRESSION_INFO to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-CompressionInfo.db
> length 0.054KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,971 BigTableZeroCopyWriter.java:213 - Block Writing component to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-CompressionInfo.db
> length 0.054KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,971 BigTableZeroCopyWriter.java:203 - Writing component FILTER to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Filter.db
> length 0.031KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,971 BigTableZeroCopyWriter.java:213 - Block Writing component to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Filter.db
> length 0.031KiB
> INFO [Stream-Deserializer-/10.214.79.62:randomport-b904af67] 2023-01-15
> 13:06:24,972 BigTableZeroCopyWriter.java:203 - Writing component SUMMARY to
> /cassandra-data/data/myawesomecluster/schema_one-randomchecksum/nb-2274426-big-Summary.db
> length 0.436KiB
> INFO
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755577#comment-17755577 ] Ekaterina Dimitrova commented on CASSANDRA-18688: - bq. Enabling anything new, like versions 11 Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out > of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.0.x > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18772) Remove dependency on commons-codec
[
https://issues.apache.org/jira/browse/CASSANDRA-18772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755571#comment-17755571
]
Ekaterina Dimitrova commented on CASSANDRA-18772:
-
{quote}this list should be carefully filtered, as some of the "unused
dependencies" may actually be used at runtime.
{quote}
I think some of those need to change probably the mvn scope from compile to
sthg else maybe? Can we open another ticket to explore this, please? I don't
want it to get lost.
{quote}assuming we clear the ML.
{quote}
I will start a thread soon
> Remove dependency on commons-codec
> --
>
> Key: CASSANDRA-18772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18772
> Project: Cassandra
> Issue Type: Bug
> Components: Dependencies
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
> The dependency doesn't seem to have any direct usage in our codebase.
> The only reference to how it might be used was found in CASSANDRA-12790.
> Considering we also deprecate in 5.0 and remove metrics-reporter-config as
> part of CASSANDRA-18743, I do not see a reason not to remove this one too.
> We should probably take the same approach as CASSANDRA-18743 - deprecate in
> 5.0 and remove in 5.1. ML thread is to be opened too.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18775) Remove libraries in lib/sigar-bin for unsupported architectures
[
https://issues.apache.org/jira/browse/CASSANDRA-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18775:
--
Description:
{code}
✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
15:41 $ ls -la lib/sigar-bin/
total 6376
drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
-rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26 libsigar-amd64-freebsd-6.so
-rw-rw-r-- 1 fermat fermat 246605 aug 17 11:26 libsigar-amd64-linux.so
-rw-rw-r-- 1 fermat fermat 251360 aug 17 11:26 libsigar-amd64-solaris.so
-rw-rw-r-- 1 fermat fermat 577452 aug 17 11:26 libsigar-ia64-hpux-11.sl
-rw-rw-r-- 1 fermat fermat 494929 aug 17 11:26 libsigar-ia64-linux.so
-rw-rw-r-- 1 fermat fermat 516096 aug 17 11:26 libsigar-pa-hpux-11.sl
-rw-rw-r-- 1 fermat fermat 425077 aug 17 11:26 libsigar-ppc64-aix-5.so
-rw-rw-r-- 1 fermat fermat 310792 aug 17 11:26 libsigar-ppc64le-linux.so
-rw-rw-r-- 1 fermat fermat 330767 aug 17 11:26 libsigar-ppc64-linux.so
-rw-rw-r-- 1 fermat fermat 400925 aug 17 11:26 libsigar-ppc-aix-5.so
-rw-rw-r-- 1 fermat fermat 258547 aug 17 11:26 libsigar-ppc-linux.so
-rw-rw-r-- 1 fermat fermat 269932 aug 17 11:26 libsigar-s390x-linux.so
-rw-rw-r-- 1 fermat fermat 261896 aug 17 11:26 libsigar-sparc64-solaris.so
-rw-rw-r-- 1 fermat fermat 285004 aug 17 11:26 libsigar-sparc-solaris.so
-rw-rw-r-- 1 fermat fermat 397440 aug 17 11:26 libsigar-universal64-macosx.dylib
-rw-rw-r-- 1 fermat fermat 377668 aug 17 11:26 libsigar-universal-macosx.dylib
-rw-rw-r-- 1 fermat fermat 179751 aug 17 11:26 libsigar-x86-freebsd-5.so
-rw-rw-r-- 1 fermat fermat 179379 aug 17 11:26 libsigar-x86-freebsd-6.so
-rw-rw-r-- 1 fermat fermat 233385 aug 17 11:26 libsigar-x86-linux.so
-rw-rw-r-- 1 fermat fermat 242880 aug 17 11:26 libsigar-x86-solaris.so
✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
15:43 $ du -shc lib/sigar-bin/
6,3Mlib/sigar-bin/
6,3Mtotal
{code}
I think we could definitely improve this. We basically need just x86-linux,
amd64-linux and two libs for macosx.
We could save like 5M from the final tarball size. From 75M down to 70M is not
bad!
Or maybe I am not getting something and we need this?
was:
{code}
✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
15:41 $ ls -la lib/sigar-bin/
total 6376
drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
-rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26 libsigar-amd64-freebsd-6.so
-rw-rw-r-- 1 fermat fermat 246605 aug 17 11:26 libsigar-amd64-linux.so
-rw-rw-r-- 1 fermat fermat 251360 aug 17 11:26 libsigar-amd64-solaris.so
-rw-rw-r-- 1 fermat fermat 577452 aug 17 11:26 libsigar-ia64-hpux-11.sl
-rw-rw-r-- 1 fermat fermat 494929 aug 17 11:26 libsigar-ia64-linux.so
-rw-rw-r-- 1 fermat fermat 516096 aug 17 11:26 libsigar-pa-hpux-11.sl
-rw-rw-r-- 1 fermat fermat 425077 aug 17 11:26 libsigar-ppc64-aix-5.so
-rw-rw-r-- 1 fermat fermat 310792 aug 17 11:26 libsigar-ppc64le-linux.so
-rw-rw-r-- 1 fermat fermat 330767 aug 17 11:26 libsigar-ppc64-linux.so
-rw-rw-r-- 1 fermat fermat 400925 aug 17 11:26 libsigar-ppc-aix-5.so
-rw-rw-r-- 1 fermat fermat 258547 aug 17 11:26 libsigar-ppc-linux.so
-rw-rw-r-- 1 fermat fermat 269932 aug 17 11:26 libsigar-s390x-linux.so
-rw-rw-r-- 1 fermat fermat 261896 aug 17 11:26 libsigar-sparc64-solaris.so
-rw-rw-r-- 1 fermat fermat 285004 aug 17 11:26 libsigar-sparc-solaris.so
-rw-rw-r-- 1 fermat fermat 397440 aug 17 11:26 libsigar-universal64-macosx.dylib
-rw-rw-r-- 1 fermat fermat 377668 aug 17 11:26 libsigar-universal-macosx.dylib
-rw-rw-r-- 1 fermat fermat 179751 aug 17 11:26 libsigar-x86-freebsd-5.so
-rw-rw-r-- 1 fermat fermat 179379 aug 17 11:26 libsigar-x86-freebsd-6.so
-rw-rw-r-- 1 fermat fermat 233385 aug 17 11:26 libsigar-x86-linux.so
-rw-rw-r-- 1 fermat fermat 242880 aug 17 11:26 libsigar-x86-solaris.so
✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
15:43 $ du -shc lib/sigar-bin/
6,3Mlib/sigar-bin/
6,3Mtotal
{code}
I think we could definitely improve this. We basically need just x86-linux,
amd64-linux and two libs for macosx.
We could save like 5M from the final tarball size. From 75M down to 70M is not
bad!
> Remove libraries in lib/sigar-bin for unsupported architectures
> ---
>
> Key: CASSANDRA-18775
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18775
> Project: Cassandra
> Issue Type: Task
>Reporter: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
>
> {code}
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:41 $ ls -la lib/sigar-bin/
> total 6376
> drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
> drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
> -rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26
[jira] [Updated] (CASSANDRA-18775) Remove libraries in lib/sigar-bin for unsupported architectures
[
https://issues.apache.org/jira/browse/CASSANDRA-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18775:
--
Fix Version/s: 5.x
> Remove libraries in lib/sigar-bin for unsupported architectures
> ---
>
> Key: CASSANDRA-18775
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18775
> Project: Cassandra
> Issue Type: Task
>Reporter: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
>
> {code}
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:41 $ ls -la lib/sigar-bin/
> total 6376
> drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
> drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
> -rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26 libsigar-amd64-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 246605 aug 17 11:26 libsigar-amd64-linux.so
> -rw-rw-r-- 1 fermat fermat 251360 aug 17 11:26 libsigar-amd64-solaris.so
> -rw-rw-r-- 1 fermat fermat 577452 aug 17 11:26 libsigar-ia64-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 494929 aug 17 11:26 libsigar-ia64-linux.so
> -rw-rw-r-- 1 fermat fermat 516096 aug 17 11:26 libsigar-pa-hpux-11.sl
> -rw-rw-r-- 1 fermat fermat 425077 aug 17 11:26 libsigar-ppc64-aix-5.so
> -rw-rw-r-- 1 fermat fermat 310792 aug 17 11:26 libsigar-ppc64le-linux.so
> -rw-rw-r-- 1 fermat fermat 330767 aug 17 11:26 libsigar-ppc64-linux.so
> -rw-rw-r-- 1 fermat fermat 400925 aug 17 11:26 libsigar-ppc-aix-5.so
> -rw-rw-r-- 1 fermat fermat 258547 aug 17 11:26 libsigar-ppc-linux.so
> -rw-rw-r-- 1 fermat fermat 269932 aug 17 11:26 libsigar-s390x-linux.so
> -rw-rw-r-- 1 fermat fermat 261896 aug 17 11:26 libsigar-sparc64-solaris.so
> -rw-rw-r-- 1 fermat fermat 285004 aug 17 11:26 libsigar-sparc-solaris.so
> -rw-rw-r-- 1 fermat fermat 397440 aug 17 11:26
> libsigar-universal64-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 377668 aug 17 11:26 libsigar-universal-macosx.dylib
> -rw-rw-r-- 1 fermat fermat 179751 aug 17 11:26 libsigar-x86-freebsd-5.so
> -rw-rw-r-- 1 fermat fermat 179379 aug 17 11:26 libsigar-x86-freebsd-6.so
> -rw-rw-r-- 1 fermat fermat 233385 aug 17 11:26 libsigar-x86-linux.so
> -rw-rw-r-- 1 fermat fermat 242880 aug 17 11:26 libsigar-x86-solaris.so
> ✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
> 15:43 $ du -shc lib/sigar-bin/
> 6,3Mlib/sigar-bin/
> 6,3Mtotal
> {code}
> I think we could definitely improve this. We basically need just x86-linux,
> amd64-linux and two libs for macosx.
> We could save like 5M from the final tarball size. From 75M down to 70M is
> not bad!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18775) Remove libraries in lib/sigar-bin for unsupported architectures
Stefan Miklosovic created CASSANDRA-18775:
-
Summary: Remove libraries in lib/sigar-bin for unsupported
architectures
Key: CASSANDRA-18775
URL: https://issues.apache.org/jira/browse/CASSANDRA-18775
Project: Cassandra
Issue Type: Task
Reporter: Stefan Miklosovic
{code}
✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
15:41 $ ls -la lib/sigar-bin/
total 6376
drwxrwxr-x 2 fermat fermat 4096 aug 17 11:26 .
drwxrwxr-x 5 fermat fermat 12288 aug 17 11:28 ..
-rw-rw-r-- 1 fermat fermat 210641 aug 17 11:26 libsigar-amd64-freebsd-6.so
-rw-rw-r-- 1 fermat fermat 246605 aug 17 11:26 libsigar-amd64-linux.so
-rw-rw-r-- 1 fermat fermat 251360 aug 17 11:26 libsigar-amd64-solaris.so
-rw-rw-r-- 1 fermat fermat 577452 aug 17 11:26 libsigar-ia64-hpux-11.sl
-rw-rw-r-- 1 fermat fermat 494929 aug 17 11:26 libsigar-ia64-linux.so
-rw-rw-r-- 1 fermat fermat 516096 aug 17 11:26 libsigar-pa-hpux-11.sl
-rw-rw-r-- 1 fermat fermat 425077 aug 17 11:26 libsigar-ppc64-aix-5.so
-rw-rw-r-- 1 fermat fermat 310792 aug 17 11:26 libsigar-ppc64le-linux.so
-rw-rw-r-- 1 fermat fermat 330767 aug 17 11:26 libsigar-ppc64-linux.so
-rw-rw-r-- 1 fermat fermat 400925 aug 17 11:26 libsigar-ppc-aix-5.so
-rw-rw-r-- 1 fermat fermat 258547 aug 17 11:26 libsigar-ppc-linux.so
-rw-rw-r-- 1 fermat fermat 269932 aug 17 11:26 libsigar-s390x-linux.so
-rw-rw-r-- 1 fermat fermat 261896 aug 17 11:26 libsigar-sparc64-solaris.so
-rw-rw-r-- 1 fermat fermat 285004 aug 17 11:26 libsigar-sparc-solaris.so
-rw-rw-r-- 1 fermat fermat 397440 aug 17 11:26 libsigar-universal64-macosx.dylib
-rw-rw-r-- 1 fermat fermat 377668 aug 17 11:26 libsigar-universal-macosx.dylib
-rw-rw-r-- 1 fermat fermat 179751 aug 17 11:26 libsigar-x86-freebsd-5.so
-rw-rw-r-- 1 fermat fermat 179379 aug 17 11:26 libsigar-x86-freebsd-6.so
-rw-rw-r-- 1 fermat fermat 233385 aug 17 11:26 libsigar-x86-linux.so
-rw-rw-r-- 1 fermat fermat 242880 aug 17 11:26 libsigar-x86-solaris.so
✔ ~/dev/cassandra/cassandra-instaclustr/cassandra [trunk L|⚑ 49]
15:43 $ du -shc lib/sigar-bin/
6,3Mlib/sigar-bin/
6,3Mtotal
{code}
I think we could definitely improve this. We basically need just x86-linux,
amd64-linux and two libs for macosx.
We could save like 5M from the final tarball size. From 75M down to 70M is not
bad!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18772) Remove dependency on commons-codec
[
https://issues.apache.org/jira/browse/CASSANDRA-18772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755541#comment-17755541
]
Maxim Muzafarov edited comment on CASSANDRA-18772 at 8/17/23 1:44 PM:
--
OK, I've found an easy way to find all the unused dependencies, but this list
should be carefully filtered, as some of the "unused dependencies" may actually
be used at runtime. I doubt we can add running of this command to the ant build
as the maven is used.
h3. The command
Tested on the latest trunk (\{{commons-codec}} is there).
{code}
# copy packages to the build/target/classes to run maven
for d in `find ./build/classes -maxdepth 2 -type d -name org -or -name com`; do
`rsync -a $d ./build/target/classes`; done
{code}
{code:java}
mvn -f ./build/apache-cassandra-5.0-SNAPSHOT.pom
org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze -Dverbose=true
-DignoreNonCompile
{code}
h3. The result
{code:java}
[WARNING] Used undeclared dependencies found:
[WARNING]io.netty:netty-handler:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-common:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-buffer:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-codec:jar:4.1.96.Final:compile
[WARNING]org.checkerframework:checker-qual:jar:3.10.0:compile
[WARNING]io.netty:netty-transport:jar:4.1.96.Final:compile
[WARNING]
io.netty:netty-transport-native-unix-common:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-transport-classes-epoll:jar:4.1.96.Final:compile
[WARNING] Unused declared dependencies found:
[WARNING]com.github.jnr:jffi:jar:native:1.3.11:compile
[WARNING]net.openhft:affinity:jar:3.23.3:compile
[WARNING]org.slf4j:log4j-over-slf4j:jar:1.7.25:compile
[WARNING]io.netty:netty-transport-native-epoll:jar:4.1.94.Final:compile
[WARNING]com.github.jnr:jnr-constants:jar:0.10.4:compile
[WARNING]io.dropwizard.metrics:metrics-logback:jar:3.1.5:compile
[WARNING]org.antlr:ST4:jar:4.0.8:compile
[WARNING]org.gridkit.jvmtool:mxdump:jar:0.14:compile
[WARNING]io.netty:netty-tcnative-boringssl-static:jar:2.0.61.Final:compile
[WARNING]com.google.j2objc:j2objc-annotations:jar:1.3:compile
[WARNING]org.gridkit.lab:jvm-attach-api:jar:1.5:compile
[WARNING]org.gridkit.jvmtool:sjk-json:jar:0.14:compile
[WARNING]org.slf4j:jcl-over-slf4j:jar:1.7.25:compile
[WARNING]com.github.jnr:jnr-ffi:jar:2.2.13:compile
[WARNING]commons-codec:commons-codec:jar:1.9:compile
[WARNING]org.caffinitas.ohc:ohc-core-j8:jar:0.5.1:compile
[WARNING]org.bouncycastle:bcpkix-jdk15on:jar:1.70:compile
[WARNING]net.java.dev.jna:jna-platform:jar:5.13.0:compile
[WARNING]com.github.jnr:jffi:jar:1.3.11:compile
[WARNING]io.netty:netty-all:jar:4.1.96.Final:compile
[WARNING]org.gridkit.jvmtool:sjk-stacktrace:jar:0.14:compile
[WARNING]
io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.94.Final:compile
[INFO]
[INFO] BUILD SUCCESS
[INFO]
{code}
EDITED:
I was confused that the netty-all library was reported in a previous attempt,
so I've rechecked everything and found the problem with that approach. The mvn
command tool requires the compiled classes to be in the {{target}} directory,
but currently they're in classes/, so once I copied them to
build/target everything started working.
was (Author: mmuzaf):
OK, I've found an easy way to find all the unused dependencies, but this list
should be carefully filtered, as some of the "unused dependencies" may actually
be used at runtime. I doubt we can add running of this command to the ant build
as the maven is used.
h3. The command
Tested on the latest trunk (\{{commons-codec}} is there).
{code}
# copy packages to the build/target/classes to run maven
for d in `find ./build/classes -maxdepth 2 -type d -name org -or -name com`; do
`rsync -a $d ./build/target/classes`; done
{code}
{code:java}
mvn -f ./build/apache-cassandra-5.0-SNAPSHOT.pom
org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze -Dverbose=true
-DignoreNonCompile
{code}
h3. The result
{code:java}
[WARNING] Used undeclared dependencies found:
[WARNING]io.netty:netty-handler:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-common:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-buffer:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-codec:jar:4.1.96.Final:compile
[WARNING]org.checkerframework:checker-qual:jar:3.10.0:compile
[WARNING]io.netty:netty-transport:jar:4.1.96.Final:compile
[WARNING]
io.netty:netty-transport-native-unix-common:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-transport-classes-epoll:jar:4.1.96.Final:compile
[WARNING] Unused declared dependencies found:
[WARNING]com.github.jnr:jffi:jar:native:1.3.11:compile
[WARNING]
[jira] [Comment Edited] (CASSANDRA-18772) Remove dependency on commons-codec
[
https://issues.apache.org/jira/browse/CASSANDRA-18772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755541#comment-17755541
]
Maxim Muzafarov edited comment on CASSANDRA-18772 at 8/17/23 1:42 PM:
--
OK, I've found an easy way to find all the unused dependencies, but this list
should be carefully filtered, as some of the "unused dependencies" may actually
be used at runtime. I doubt we can add running of this command to the ant build
as the maven is used.
h3. The command
Tested on the latest trunk (\{{commons-codec}} is there).
{code}
# copy packages to the build/target/classes to run maven
for d in `find ./build/classes -maxdepth 2 -type d -name org -or -name com`; do
`rsync -a $d ./build/target/classes`; done
{code}
{code:java}
mvn -f ./build/apache-cassandra-5.0-SNAPSHOT.pom
org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze -Dverbose=true
-DignoreNonCompile
{code}
h3. The result
{code:java}
[WARNING] Used undeclared dependencies found:
[WARNING]io.netty:netty-handler:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-common:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-buffer:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-codec:jar:4.1.96.Final:compile
[WARNING]org.checkerframework:checker-qual:jar:3.10.0:compile
[WARNING]io.netty:netty-transport:jar:4.1.96.Final:compile
[WARNING]
io.netty:netty-transport-native-unix-common:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-transport-classes-epoll:jar:4.1.96.Final:compile
[WARNING] Unused declared dependencies found:
[WARNING]com.github.jnr:jffi:jar:native:1.3.11:compile
[WARNING]net.openhft:affinity:jar:3.23.3:compile
[WARNING]org.slf4j:log4j-over-slf4j:jar:1.7.25:compile
[WARNING]io.netty:netty-transport-native-epoll:jar:4.1.94.Final:compile
[WARNING]com.github.jnr:jnr-constants:jar:0.10.4:compile
[WARNING]io.dropwizard.metrics:metrics-logback:jar:3.1.5:compile
[WARNING]org.antlr:ST4:jar:4.0.8:compile
[WARNING]org.gridkit.jvmtool:mxdump:jar:0.14:compile
[WARNING]io.netty:netty-tcnative-boringssl-static:jar:2.0.61.Final:compile
[WARNING]com.google.j2objc:j2objc-annotations:jar:1.3:compile
[WARNING]org.gridkit.lab:jvm-attach-api:jar:1.5:compile
[WARNING]org.gridkit.jvmtool:sjk-json:jar:0.14:compile
[WARNING]org.slf4j:jcl-over-slf4j:jar:1.7.25:compile
[WARNING]com.github.jnr:jnr-ffi:jar:2.2.13:compile
[WARNING]commons-codec:commons-codec:jar:1.9:compile
[WARNING]org.caffinitas.ohc:ohc-core-j8:jar:0.5.1:compile
[WARNING]org.bouncycastle:bcpkix-jdk15on:jar:1.70:compile
[WARNING]net.java.dev.jna:jna-platform:jar:5.13.0:compile
[WARNING]com.github.jnr:jffi:jar:1.3.11:compile
[WARNING]io.netty:netty-all:jar:4.1.96.Final:compile
[WARNING]org.gridkit.jvmtool:sjk-stacktrace:jar:0.14:compile
[WARNING]
io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.94.Final:compile
[INFO]
{code}
EDITED:
I was confused that the netty-all library was reported in a previous attempt,
so I've rechecked everything and found the problem with that approach. The mvn
command tool requires the compiled classes to be in the {{target}} directory,
but currently they're in classes/, so once I copied them to
build/target everything started working.
was (Author: mmuzaf):
OK, I've found an easy way to find all the unused dependencies, but this list
should be carefully filtered, as some of the "unused dependencies" may actually
be used at runtime. I doubt we can add running of this command to the ant build
as the maven is used.
h3. The command
Tested on the latest trunk (\{{commons-codec}} is there).
{code:java}
mvn -f ./build/apache-cassandra-5.0-SNAPSHOT.pom
org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze -Dverbose=true
-DignoreNonCompile
{code}
h3. The result
{code:java}
[WARNING] Used undeclared dependencies found:
[WARNING]io.netty:netty-handler:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-common:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-buffer:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-codec:jar:4.1.96.Final:compile
[WARNING]org.checkerframework:checker-qual:jar:3.10.0:compile
[WARNING]io.netty:netty-transport:jar:4.1.96.Final:compile
[WARNING]
io.netty:netty-transport-native-unix-common:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-transport-classes-epoll:jar:4.1.96.Final:compile
[WARNING] Unused declared dependencies found:
[WARNING]com.github.jnr:jffi:jar:native:1.3.11:compile
[WARNING]net.openhft:affinity:jar:3.23.3:compile
[WARNING]org.slf4j:log4j-over-slf4j:jar:1.7.25:compile
[WARNING]io.netty:netty-transport-native-epoll:jar:4.1.94.Final:compile
[WARNING]com.github.jnr:jnr-constants:jar:0.10.4:compile
[WARNING]
[jira] [Updated] (CASSANDRA-18739) UDF functions fail to load on rolling restart
[
https://issues.apache.org/jira/browse/CASSANDRA-18739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18739:
--
Status: Review In Progress (was: Changes Suggested)
> UDF functions fail to load on rolling restart
> -
>
> Key: CASSANDRA-18739
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18739
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/UDF
>Reporter: Claude Warren
>Assignee: Claude Warren
>Priority: High
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Attachments: udf_error.cql, udf_error_data.cql
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> UDFs fail to reload properly after a rolling restart.
> h3. *Symptom:*
> NPE thrown when used after restart.
> h3. *Steps to recreate:*
> # Create a cluster as per cql file
> # Populate the cluster with data.cql.
> # Execute SELECT city_measurements(city, measurement, 16.5) AS m FROM current
> # expect min and max values for cities.
> # Performing a rolling restart on one server.
> # When the server is back up
> # Execute SELECT city_measurements(city, measurement, 16.5) AS m FROM current
> # expect: error result with NPE message.
> {*}Analysis{*}:
> During system restart the SchemaKeyspace.fetchNonSystemKeyspaces() is called,
> when a keyspace with a UDF is loaded the SchemaKeyspace method
> createUDFFromRow() is called, this in turn calls UDFunction.create() which
> eventually calls back to UDFunction constructor where the
> Schema.instance.getKeyspaceMetadata() is called with the keyspace for the UDF
> name as the argument. However, the keyspace for the UDF name is being
> constructed and is not yet in the instance so the method returns null for the
> KeyspaceMetadata. That null KeyspaceMetadata is then used in the udfContext.
> Later when the UDF method is called, if there is a need to call a method on
> the keyspaceMetadata, such as udfContext.newUDTValue() where the
> implementation uses keyspaceMetadata.types, a null pointer is thrown.
> I have verified this affects version 4.0, 4.1 and trunk. I have not verified
> 3.x but I suspect it is the same there.
> I modified UDFunction constructor to assert that the metadata was not null
> and received the following stack trace
> ERROR [main] 2023-08-09 11:44:46,408 CassandraDaemon.java:911 - Exception
> encountered during startup
> java.lang.AssertionError: No metadata for temperatures.city_measurements_sfunc
> at
> org.apache.cassandra.cql3.functions.UDFunction.(UDFunction.java:240)
> at
> org.apache.cassandra.cql3.functions.JavaBasedUDFunction.(JavaBasedUDFunction.java:195)
> at
> org.apache.cassandra.cql3.functions.UDFunction.create(UDFunction.java:276)
> at
> org.apache.cassandra.schema.SchemaKeyspace.createUDFFromRow(SchemaKeyspace.java:1182)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchUDFs(SchemaKeyspace.java:1131)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchFunctions(SchemaKeyspace.java:1119)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:859)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:848)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:836)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:132)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:121)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:287)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:765)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:889)
>
> {{*Possible solution:*}}
> *Version 4.x*
> Create a KeyspaceMetadata.Builder class that uses accepts the types, tables
> and views but uses a builder for the functions.
> Add a KeyspaceMetadata constructor to accept the KeyspaceMetadata.Builder so
> that the function builder keyspaceMetadata value can be set correctly during
> construction of the KeyspaceMetadata.
> Modify SchemaKeyspace.fetchKeyspace(string) so that it uses the
> KeyspaceMetadata.Builder.
>
> *Version 5.x*
> Similar to 4.x except that the KeyspaceMetadata.Builder will have to have
> builders for Views and Tables because the functions necessary to construct
> those objects will not be available until the KeyspaceMetadata.Builder
> constructs it.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18739) UDF functions fail to load on rolling restart
[
https://issues.apache.org/jira/browse/CASSANDRA-18739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1774#comment-1774
]
Stefan Miklosovic commented on CASSANDRA-18739:
---
Thanks [~claude], I incorporated your test simplification and other
refactorings into 5.0 branch.
[4.0 PR|https://github.com/apache/cassandra/pull/2584] Claude's patch
[4.1 PR|https://github.com/apache/cassandra/pull/2585] Claude's patch
[5.0 PR|https://github.com/apache/cassandra/pull/2597] my patch same as Claude's
[~adelapena] are you happy with this? I will approach the builds then.
> UDF functions fail to load on rolling restart
> -
>
> Key: CASSANDRA-18739
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18739
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/UDF
>Reporter: Claude Warren
>Assignee: Claude Warren
>Priority: High
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Attachments: udf_error.cql, udf_error_data.cql
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> UDFs fail to reload properly after a rolling restart.
> h3. *Symptom:*
> NPE thrown when used after restart.
> h3. *Steps to recreate:*
> # Create a cluster as per cql file
> # Populate the cluster with data.cql.
> # Execute SELECT city_measurements(city, measurement, 16.5) AS m FROM current
> # expect min and max values for cities.
> # Performing a rolling restart on one server.
> # When the server is back up
> # Execute SELECT city_measurements(city, measurement, 16.5) AS m FROM current
> # expect: error result with NPE message.
> {*}Analysis{*}:
> During system restart the SchemaKeyspace.fetchNonSystemKeyspaces() is called,
> when a keyspace with a UDF is loaded the SchemaKeyspace method
> createUDFFromRow() is called, this in turn calls UDFunction.create() which
> eventually calls back to UDFunction constructor where the
> Schema.instance.getKeyspaceMetadata() is called with the keyspace for the UDF
> name as the argument. However, the keyspace for the UDF name is being
> constructed and is not yet in the instance so the method returns null for the
> KeyspaceMetadata. That null KeyspaceMetadata is then used in the udfContext.
> Later when the UDF method is called, if there is a need to call a method on
> the keyspaceMetadata, such as udfContext.newUDTValue() where the
> implementation uses keyspaceMetadata.types, a null pointer is thrown.
> I have verified this affects version 4.0, 4.1 and trunk. I have not verified
> 3.x but I suspect it is the same there.
> I modified UDFunction constructor to assert that the metadata was not null
> and received the following stack trace
> ERROR [main] 2023-08-09 11:44:46,408 CassandraDaemon.java:911 - Exception
> encountered during startup
> java.lang.AssertionError: No metadata for temperatures.city_measurements_sfunc
> at
> org.apache.cassandra.cql3.functions.UDFunction.(UDFunction.java:240)
> at
> org.apache.cassandra.cql3.functions.JavaBasedUDFunction.(JavaBasedUDFunction.java:195)
> at
> org.apache.cassandra.cql3.functions.UDFunction.create(UDFunction.java:276)
> at
> org.apache.cassandra.schema.SchemaKeyspace.createUDFFromRow(SchemaKeyspace.java:1182)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchUDFs(SchemaKeyspace.java:1131)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchFunctions(SchemaKeyspace.java:1119)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:859)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:848)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:836)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:132)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:121)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:287)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:765)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:889)
>
> {{*Possible solution:*}}
> *Version 4.x*
> Create a KeyspaceMetadata.Builder class that uses accepts the types, tables
> and views but uses a builder for the functions.
> Add a KeyspaceMetadata constructor to accept the KeyspaceMetadata.Builder so
> that the function builder keyspaceMetadata value can be set correctly during
> construction of the KeyspaceMetadata.
> Modify SchemaKeyspace.fetchKeyspace(string) so that it uses the
> KeyspaceMetadata.Builder.
>
> *Version 5.x*
> Similar to 4.x except that the KeyspaceMetadata.Builder will have to have
> builders for Views and Tables because the functions necessary to construct
> those objects will not be available until
[cassandra-website] branch asf-staging updated (698541f7 -> 7a90dc37)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 698541f7 generate docs for cceb2c06 new 7a90dc37 generate docs for cceb2c06 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (698541f7) \ N -- N -- N refs/heads/asf-staging (7a90dc37) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4852435 -> 4852435 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18772) Remove dependency on commons-codec
[
https://issues.apache.org/jira/browse/CASSANDRA-18772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755541#comment-17755541
]
Maxim Muzafarov edited comment on CASSANDRA-18772 at 8/17/23 1:23 PM:
--
OK, I've found an easy way to find all the unused dependencies, but this list
should be carefully filtered, as some of the "unused dependencies" may actually
be used at runtime. I doubt we can add running of this command to the ant build
as the maven is used.
h3. The command
Tested on the latest trunk (\{{commons-codec}} is there).
{code:java}
mvn -f ./build/apache-cassandra-5.0-SNAPSHOT.pom
org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze -Dverbose=true
-DignoreNonCompile
{code}
h3. The result
{code:java}
[WARNING] Used undeclared dependencies found:
[WARNING]io.netty:netty-handler:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-common:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-buffer:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-codec:jar:4.1.96.Final:compile
[WARNING]org.checkerframework:checker-qual:jar:3.10.0:compile
[WARNING]io.netty:netty-transport:jar:4.1.96.Final:compile
[WARNING]
io.netty:netty-transport-native-unix-common:jar:4.1.96.Final:compile
[WARNING]io.netty:netty-transport-classes-epoll:jar:4.1.96.Final:compile
[WARNING] Unused declared dependencies found:
[WARNING]com.github.jnr:jffi:jar:native:1.3.11:compile
[WARNING]net.openhft:affinity:jar:3.23.3:compile
[WARNING]org.slf4j:log4j-over-slf4j:jar:1.7.25:compile
[WARNING]io.netty:netty-transport-native-epoll:jar:4.1.94.Final:compile
[WARNING]com.github.jnr:jnr-constants:jar:0.10.4:compile
[WARNING]io.dropwizard.metrics:metrics-logback:jar:3.1.5:compile
[WARNING]org.antlr:ST4:jar:4.0.8:compile
[WARNING]org.gridkit.jvmtool:mxdump:jar:0.14:compile
[WARNING]io.netty:netty-tcnative-boringssl-static:jar:2.0.61.Final:compile
[WARNING]com.google.j2objc:j2objc-annotations:jar:1.3:compile
[WARNING]org.gridkit.lab:jvm-attach-api:jar:1.5:compile
[WARNING]org.gridkit.jvmtool:sjk-json:jar:0.14:compile
[WARNING]org.slf4j:jcl-over-slf4j:jar:1.7.25:compile
[WARNING]com.github.jnr:jnr-ffi:jar:2.2.13:compile
[WARNING]commons-codec:commons-codec:jar:1.9:compile
[WARNING]org.caffinitas.ohc:ohc-core-j8:jar:0.5.1:compile
[WARNING]org.bouncycastle:bcpkix-jdk15on:jar:1.70:compile
[WARNING]net.java.dev.jna:jna-platform:jar:5.13.0:compile
[WARNING]com.github.jnr:jffi:jar:1.3.11:compile
[WARNING]io.netty:netty-all:jar:4.1.96.Final:compile
[WARNING]org.gridkit.jvmtool:sjk-stacktrace:jar:0.14:compile
[WARNING]
io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.94.Final:compile
[INFO]
{code}
EDITED:
I was confused that the netty-all library was reported in a previous attempt,
so I've rechecked everything and found the problem with that approach. The mvn
command tool requires the compiled classes to be in the {{target}} directory,
but currently they're in classes/, so once I copied them to
build/target everything started working.
was (Author: mmuzaf):
OK, I've found an easy way to find all the unused dependencies, but this list
should be carefully filtered, as some of the "unused dependencies" may actually
be used at runtime. I doubt we can add running of this command to the ant build
as the maven is used.
h3. The command
Tested on the latest trunk (\{{commons-codec}} is there).
{code:java}
mvn -f ./build/apache-cassandra-5.0-SNAPSHOT.pom
org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze -Dverbose=true
-DignoreNonCompile
{code}
h3. The result
{code:java}
[INFO] --- dependency:3.2.0:analyze (default-cli) @ cassandra-all ---
[WARNING] Unused declared dependencies found:
[WARNING]com.github.jnr:jffi:jar:native:1.3.11:compile
[WARNING]net.openhft:chronicle-threads:jar:2.23.25:compile
[WARNING]org.psjava:psjava:jar:0.1.19:compile
[WARNING]ch.qos.logback:logback-core:jar:1.2.9:compile
[WARNING]io.netty:netty-transport-native-epoll:jar:4.1.94.Final:compile
[WARNING]org.lz4:lz4-java:jar:1.8.0:compile
[WARNING]org.apache.commons:commons-lang3:jar:3.11:compile
[WARNING]com.fasterxml.jackson.core:jackson-annotations:jar:2.13.2:compile
[WARNING]net.openhft:chronicle-bytes:jar:2.23.33:compile
[WARNING]ch.obermuhlner:big-math:jar:2.3.0:compile
[WARNING]io.airlift:airline:jar:0.8:compile
[WARNING]io.dropwizard.metrics:metrics-logback:jar:3.1.5:compile
[WARNING]io.dropwizard.metrics:metrics-core:jar:3.1.5:compile
[WARNING]org.gridkit.jvmtool:sjk-core:jar:0.14:compile
[WARNING]com.github.seancfoley:ipaddress:jar:5.3.3:compile
[WARNING]com.google.j2objc:j2objc-annotations:jar:1.3:compile
[WARNING]com.fasterxml.jackson.core:jackson-databind:jar:2.13.2.2:compile
[jira] [Updated] (CASSANDRA-18760) Need to block the ability to DROP table column of 'map' type and add back as 'blob' type
[
https://issues.apache.org/jira/browse/CASSANDRA-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18760:
--
Fix Version/s: 3.0.x
3.11.x
> Need to block the ability to DROP table column of 'map' type and add back as
> 'blob' type
>
>
> Key: CASSANDRA-18760
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18760
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Paul Ayers
>Assignee: Stefan Miklosovic
>Priority: Urgent
> Labels: 4.0.10
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Recently hit an uncoverable situation in Cassandra 4.0.10 after dropping a
> 'map' column and adding it back as a 'blob', which caused corruption that
> neither offline nor online scrub could fix.
> When dropping a 'blob' column and attempting to add it back as a 'map' type,
> the operation is blocked with:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> re-add previously dropped column 'col1' of type map,
> incompatible with previous type blob"
> {code}
> We need to do the same going from 'map' to 'blob' to avoid this potentially
> very bad scenario that can cause data loss.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18580) Baseline Metrics for Accord Transactions
[
https://issues.apache.org/jira/browse/CASSANDRA-18580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacek Lewandowski updated CASSANDRA-18580:
--
Test and Documentation Plan: run tests
Status: Patch Available (was: In Progress)
> Baseline Metrics for Accord Transactions
>
>
> Key: CASSANDRA-18580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18580
> Project: Cassandra
> Issue Type: Improvement
> Components: Accord, Observability/JMX, Observability/Metrics
>Reporter: Caleb Rackliffe
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Based on some conversations w/ [~benedict] and [~dcapwell], this is the
> initial set of metrics that seem both feasible to implement and useful as we
> monitor the health of a cluster performing Accord transactions:
> 1.) Basic latency metrics for transactions up to the point of COMMIT and rate
> metrics for preemption, failure, and timeouts at the coordinator.
> This has already been implemented and split into read and write-specific
> metrics. Our position for now is that metrics around preemption should be
> useful in place of a more difficult-to-define metric around how many
> transactions are completed via recovery.
> 2.) Global cache stats/metrics (i.e. aggregated for all command stores)
> We could, at some point, build metrics scoped to a specific {{CommandStore}},
> but they might be awkward in MBean/JMX space, as command stores would have to
> be identified by ID or key range…the latter possibly being able to change
> across epochs. (An alternative would be just publishing command
> store-specific stats on-demand to a virtual table instead.)
> 3.) Something like a decaying histogram of the number of dependencies per
> transaction (or per partial transaction).
> If this is getting worse over time, it could be useful to know/be a way for
> us to detect that contention is increasing. We should be able to hook this up
> to {{ProgressLog}} notifications. Recording for PartialDeps/PartialTxn (which
> ProgressLog gives us at pre-accept) seems acceptable, given this is a
> directional metric.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18760) Need to block the ability to DROP table column of 'map' type and add back as 'blob' type
[
https://issues.apache.org/jira/browse/CASSANDRA-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755538#comment-17755538
]
Stefan Miklosovic edited comment on CASSANDRA-18760 at 8/17/23 1:20 PM:
We identified that this is not happening in 4.1+. The reason is that there was
CASSANDRA-16905 added but it was not added to 4.0.
We are going to backport CASSANDRA-16905 to 4.0 and add more tests. Then in
4.1+ we just add tests.
3.0 patch https://github.com/apache/cassandra/pull/2608
3.11 patch https://github.com/apache/cassandra/pull/2607
4.0 patch https://github.com/apache/cassandra/pull/2600
4.1 patch https://github.com/apache/cassandra/pull/2602
I expect 5.0 and trunk patches to be straightforward, basically just tests from
4.1 applied.
I ll do the builds once this is all good for everybody.
was (Author: smiklosovic):
We identified that this is not happening in 4.1+. The reason is that there was
CASSANDRA-16905 added but it was not added to 4.0.
We are going to backport CASSANDRA-16905 to 4.0 and add more tests. Then in
4.1+ we just add tests.
4.0 patch https://github.com/apache/cassandra/pull/2600
4.1 patch https://github.com/apache/cassandra/pull/2602
I expect 5.0 and trunk patches to be straightforward, basically just tests from
4.1 applied.
I ll do the builds once this is all good for everybody.
> Need to block the ability to DROP table column of 'map' type and add back as
> 'blob' type
>
>
> Key: CASSANDRA-18760
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18760
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Paul Ayers
>Assignee: Stefan Miklosovic
>Priority: Urgent
> Labels: 4.0.10
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Recently hit an uncoverable situation in Cassandra 4.0.10 after dropping a
> 'map' column and adding it back as a 'blob', which caused corruption that
> neither offline nor online scrub could fix.
> When dropping a 'blob' column and attempting to add it back as a 'map' type,
> the operation is blocked with:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> re-add previously dropped column 'col1' of type map,
> incompatible with previous type blob"
> {code}
> We need to do the same going from 'map' to 'blob' to avoid this potentially
> very bad scenario that can cause data loss.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18772) Remove dependency on commons-codec
[
https://issues.apache.org/jira/browse/CASSANDRA-18772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755541#comment-17755541
]
Maxim Muzafarov commented on CASSANDRA-18772:
-
OK, I've found an easy way to find all the unused dependencies, but this list
should be carefully filtered, as some of the "unused dependencies" may actually
be used at runtime. I doubt we can add running of this command to the ant build
as the maven is used.
h3. The command
Tested on the latest trunk (\{{commons-codec}} is there).
{code:java}
mvn -f ./build/apache-cassandra-5.0-SNAPSHOT.pom
org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze -Dverbose=true
-DignoreNonCompile
{code}
h3. The result
{code:java}
[INFO] --- dependency:3.2.0:analyze (default-cli) @ cassandra-all ---
[WARNING] Unused declared dependencies found:
[WARNING]com.github.jnr:jffi:jar:native:1.3.11:compile
[WARNING]net.openhft:chronicle-threads:jar:2.23.25:compile
[WARNING]org.psjava:psjava:jar:0.1.19:compile
[WARNING]ch.qos.logback:logback-core:jar:1.2.9:compile
[WARNING]io.netty:netty-transport-native-epoll:jar:4.1.94.Final:compile
[WARNING]org.lz4:lz4-java:jar:1.8.0:compile
[WARNING]org.apache.commons:commons-lang3:jar:3.11:compile
[WARNING]com.fasterxml.jackson.core:jackson-annotations:jar:2.13.2:compile
[WARNING]net.openhft:chronicle-bytes:jar:2.23.33:compile
[WARNING]ch.obermuhlner:big-math:jar:2.3.0:compile
[WARNING]io.airlift:airline:jar:0.8:compile
[WARNING]io.dropwizard.metrics:metrics-logback:jar:3.1.5:compile
[WARNING]io.dropwizard.metrics:metrics-core:jar:3.1.5:compile
[WARNING]org.gridkit.jvmtool:sjk-core:jar:0.14:compile
[WARNING]com.github.seancfoley:ipaddress:jar:5.3.3:compile
[WARNING]com.google.j2objc:j2objc-annotations:jar:1.3:compile
[WARNING]com.fasterxml.jackson.core:jackson-databind:jar:2.13.2.2:compile
[WARNING]net.java.dev.jna:jna:jar:5.13.0:compile
[WARNING]org.gridkit.lab:jvm-attach-api:jar:1.5:compile
[WARNING]org.gridkit.jvmtool:sjk-json:jar:0.14:compile
[WARNING]org.agrona:agrona:jar:1.17.1:compile
[WARNING]org.slf4j:jcl-over-slf4j:jar:1.7.25:compile
[WARNING]
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.13.2:compile
[WARNING]com.github.jbellis:jamm:jar:0.4.0:compile
[WARNING]net.openhft:posix:jar:2.24ea4:compile
[WARNING]net.openhft:chronicle-queue:jar:5.23.37:compile
[WARNING]commons-codec:commons-codec:jar:1.9:compile
[WARNING]com.boundary:high-scale-lib:jar:1.0.6:compile
[WARNING]org.caffinitas.ohc:ohc-core-j8:jar:0.5.1:compile
[WARNING]com.addthis.metrics:reporter-config3:jar:3.0.3:compile
[WARNING]
com.datastax.cassandra:cassandra-driver-core:jar:shaded:3.11.0:compile
[WARNING]com.github.luben:zstd-jni:jar:1.5.5-1:compile
[WARNING]com.carrotsearch:hppc:jar:0.8.1:compile
[WARNING]org.xerial.snappy:snappy-java:jar:1.1.10.1:compile
[WARNING]org.bouncycastle:bcpkix-jdk15on:jar:1.70:compile
[WARNING]com.google.code.findbugs:jsr305:jar:2.0.2:compile
[WARNING]com.clearspring.analytics:stream:jar:2.5.2:compile
[WARNING]org.mindrot:jbcrypt:jar:0.4:compile
[WARNING]
io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.94.Final:compile
[WARNING]ch.qos.logback:logback-classic:jar:1.2.9:compile
[WARNING]com.google.guava:guava:jar:32.0.1-jre:compile
[WARNING]org.apache.commons:commons-math3:jar:3.2:compile
[WARNING]org.yaml:snakeyaml:jar:1.26:compile
[WARNING]org.apache.lucene:lucene-core:jar:9.7.0:compile
[WARNING]net.openhft:affinity:jar:3.23.3:compile
[WARNING]org.eclipse.jdt:ecj:jar:3.33.0:compile
[WARNING]org.slf4j:log4j-over-slf4j:jar:1.7.25:compile
[WARNING]org.caffinitas.ohc:ohc-core:jar:0.5.1:compile
[WARNING]org.gridkit.jvmtool:sjk-cli:jar:0.14:compile
[WARNING]org.apache.lucene:lucene-analysis-common:jar:9.7.0:compile
[WARNING]com.github.jnr:jnr-constants:jar:0.10.4:compile
[WARNING]org.hdrhistogram:HdrHistogram:jar:2.1.9:compile
[WARNING]net.openhft:chronicle-core:jar:2.23.36:compile
[WARNING]com.github.ben-manes.caffeine:caffeine:jar:2.9.2:compile
[WARNING]org.antlr:ST4:jar:4.0.8:compile
[WARNING]org.gridkit.jvmtool:mxdump:jar:0.14:compile
[WARNING]io.netty:netty-tcnative-boringssl-static:jar:2.0.61.Final:compile
[WARNING]org.antlr:antlr-runtime:jar:3.5.2:compile
[WARNING]com.beust:jcommander:jar:1.30:compile
[WARNING]com.fasterxml.jackson.core:jackson-core:jar:2.13.2:compile
[WARNING]javax.inject:javax.inject:jar:1:compile
[WARNING]io.dropwizard.metrics:metrics-jvm:jar:3.1.5:compile
[WARNING]commons-cli:commons-cli:jar:1.5.0:compile
[WARNING]com.github.jnr:jnr-ffi:jar:2.2.13:compile
[WARNING]org.slf4j:slf4j-api:jar:1.7.25:compile
[WARNING]org.fusesource:sigar:jar:1.6.4:compile
[WARNING]org.jctools:jctools-core:jar:3.1.0:compile
[WARNING]
[jira] [Commented] (CASSANDRA-18760) Need to block the ability to DROP table column of 'map' type and add back as 'blob' type
[
https://issues.apache.org/jira/browse/CASSANDRA-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755538#comment-17755538
]
Stefan Miklosovic commented on CASSANDRA-18760:
---
We identified that this is not happening in 4.1+. The reason is that there was
CASSANDRA-16905 added but it was not added to 4.0.
We are going to backport CASSANDRA-16905 to 4.0 and add more tests. Then in
4.1+ we just add tests.
4.0 patch https://github.com/apache/cassandra/pull/2600
4.1 patch https://github.com/apache/cassandra/pull/2602
I expect 5.0 and trunk patches to be straightforward, basically just tests from
4.1 applied.
I ll do the builds once this is all good for everybody.
> Need to block the ability to DROP table column of 'map' type and add back as
> 'blob' type
>
>
> Key: CASSANDRA-18760
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18760
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Paul Ayers
>Assignee: Stefan Miklosovic
>Priority: Urgent
> Labels: 4.0.10
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Recently hit an uncoverable situation in Cassandra 4.0.10 after dropping a
> 'map' column and adding it back as a 'blob', which caused corruption that
> neither offline nor online scrub could fix.
> When dropping a 'blob' column and attempting to add it back as a 'map' type,
> the operation is blocked with:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> re-add previously dropped column 'col1' of type map,
> incompatible with previous type blob"
> {code}
> We need to do the same going from 'map' to 'blob' to avoid this potentially
> very bad scenario that can cause data loss.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18585) Alter Type does not validate changes like Create Type does
[
https://issues.apache.org/jira/browse/CASSANDRA-18585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755532#comment-17755532
]
Andres de la Peña commented on CASSANDRA-18585:
---
Looks good to me, except for a nit on the 4.0 PR.
Shouldn't we apply the fix to {{3.0}} and {{3.11}} too? I think those branches
are supported until 5.0 is released.
> Alter Type does not validate changes like Create Type does
> --
>
> Key: CASSANDRA-18585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18585
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: David Capwell
>Assignee: Roman Mushchinski
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Create Type attempts to block undesired field types, but this validation is
> not cared over to Alter Type Add Field; which allows you to add
> unexpected/desired types
> {code}
> Assertions.assertThatThrownBy(() -> createType("CREATE TYPE %s (f
> counter)")).hasRootCauseMessage("A user type cannot contain counters");
> String type = createType(KEYSPACE, "CREATE TYPE %s (a int)");
> schemaChange(String.format("ALTER TYPE %s.%s ADD f counter", KEYSPACE, type));
> UserType udt =
> Keyspace.open(KEYSPACE).getMetadata().types.get(UTF8Type.instance.decompose(type)).get();
> logger.warn("UDT: {}", udt);
> {code}
> {code}
> UDT:
> org.apache.cassandra.db.marshal.UserType(cql_test_keyspace,747970655f3031,61:org.apache.cassandra.db.marshal.Int32Type,66:org.apache.cassandra.db.marshal.CounterColumnType)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18739) UDF functions fail to load on rolling restart
[
https://issues.apache.org/jira/browse/CASSANDRA-18739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755520#comment-17755520
]
Claude Warren commented on CASSANDRA-18739:
---
Further analysis shows the issue is in non-system keyspaces, where there is a
user defined type (UDT) and a UDF that may create the UDT using the
udfContxt.newUTDValue() method. Alternatively calling as UDF calling
udfContext().newTupleValue() method will also trigger the NPE.
The issue is that the keyspace variable in udfContextImpl is set to null
because the keyspace is being created when the value is set.
There is no requirement for more than one server.
> UDF functions fail to load on rolling restart
> -
>
> Key: CASSANDRA-18739
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18739
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/UDF
>Reporter: Claude Warren
>Assignee: Claude Warren
>Priority: High
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Attachments: udf_error.cql, udf_error_data.cql
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> UDFs fail to reload properly after a rolling restart.
> h3. *Symptom:*
> NPE thrown when used after restart.
> h3. *Steps to recreate:*
> # Create a cluster as per cql file
> # Populate the cluster with data.cql.
> # Execute SELECT city_measurements(city, measurement, 16.5) AS m FROM current
> # expect min and max values for cities.
> # Performing a rolling restart on one server.
> # When the server is back up
> # Execute SELECT city_measurements(city, measurement, 16.5) AS m FROM current
> # expect: error result with NPE message.
> {*}Analysis{*}:
> During system restart the SchemaKeyspace.fetchNonSystemKeyspaces() is called,
> when a keyspace with a UDF is loaded the SchemaKeyspace method
> createUDFFromRow() is called, this in turn calls UDFunction.create() which
> eventually calls back to UDFunction constructor where the
> Schema.instance.getKeyspaceMetadata() is called with the keyspace for the UDF
> name as the argument. However, the keyspace for the UDF name is being
> constructed and is not yet in the instance so the method returns null for the
> KeyspaceMetadata. That null KeyspaceMetadata is then used in the udfContext.
> Later when the UDF method is called, if there is a need to call a method on
> the keyspaceMetadata, such as udfContext.newUDTValue() where the
> implementation uses keyspaceMetadata.types, a null pointer is thrown.
> I have verified this affects version 4.0, 4.1 and trunk. I have not verified
> 3.x but I suspect it is the same there.
> I modified UDFunction constructor to assert that the metadata was not null
> and received the following stack trace
> ERROR [main] 2023-08-09 11:44:46,408 CassandraDaemon.java:911 - Exception
> encountered during startup
> java.lang.AssertionError: No metadata for temperatures.city_measurements_sfunc
> at
> org.apache.cassandra.cql3.functions.UDFunction.(UDFunction.java:240)
> at
> org.apache.cassandra.cql3.functions.JavaBasedUDFunction.(JavaBasedUDFunction.java:195)
> at
> org.apache.cassandra.cql3.functions.UDFunction.create(UDFunction.java:276)
> at
> org.apache.cassandra.schema.SchemaKeyspace.createUDFFromRow(SchemaKeyspace.java:1182)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchUDFs(SchemaKeyspace.java:1131)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchFunctions(SchemaKeyspace.java:1119)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:859)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:848)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:836)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:132)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:121)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:287)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:765)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:889)
>
> {{*Possible solution:*}}
> *Version 4.x*
> Create a KeyspaceMetadata.Builder class that uses accepts the types, tables
> and views but uses a builder for the functions.
> Add a KeyspaceMetadata constructor to accept the KeyspaceMetadata.Builder so
> that the function builder keyspaceMetadata value can be set correctly during
> construction of the KeyspaceMetadata.
> Modify SchemaKeyspace.fetchKeyspace(string) so that it uses the
> KeyspaceMetadata.Builder.
>
> *Version 5.x*
> Similar to 4.x except that the KeyspaceMetadata.Builder will have to have
> builders for Views and Tables because the functions
[jira] [Updated] (CASSANDRA-18760) Need to block the ability to DROP table column of 'map' type and add back as 'blob' type
[
https://issues.apache.org/jira/browse/CASSANDRA-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-18760:
--
Reviewers: Andres de la Peña, Andres de la Peña (was: Andres de la Peña)
Andres de la Peña, Andres de la Peña (was: Andres de la Peña)
Status: Review In Progress (was: Patch Available)
> Need to block the ability to DROP table column of 'map' type and add back as
> 'blob' type
>
>
> Key: CASSANDRA-18760
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18760
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Paul Ayers
>Assignee: Stefan Miklosovic
>Priority: Urgent
> Labels: 4.0.10
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Recently hit an uncoverable situation in Cassandra 4.0.10 after dropping a
> 'map' column and adding it back as a 'blob', which caused corruption that
> neither offline nor online scrub could fix.
> When dropping a 'blob' column and attempting to add it back as a 'map' type,
> the operation is blocked with:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> re-add previously dropped column 'col1' of type map,
> incompatible with previous type blob"
> {code}
> We need to do the same going from 'map' to 'blob' to avoid this potentially
> very bad scenario that can cause data loss.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18770) Open File Descriptors not cleared post upgrade from 3.11.9 to 4.0.5
[
https://issues.apache.org/jira/browse/CASSANDRA-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755511#comment-17755511
]
Pere Balaguer commented on CASSANDRA-18770:
---
We faced a similar issue in >3.x .. <4.0.9, solved after upgrading to 4.0.9.
See https://issues.apache.org/jira/browse/CASSANDRA-18176
> Open File Descriptors not cleared post upgrade from 3.11.9 to 4.0.5
> ---
>
> Key: CASSANDRA-18770
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18770
> Project: Cassandra
> Issue Type: Bug
>Reporter: vaibhav khedkar
>Priority: Normal
>
> We recently upgraded our fleet of ~2500 Cassandra instances from 3.11.9 to
> 4.0.5.
> After the upgrade, we are seeing a unique issue where the compacted
> SSTables's file descriptors are still present and are never cleared. This is
> causing false disk alerts. We have to restart nodes very often in order to
> reclaim the disk space.
> We use the *Apache* version of C* along with CentOS Linux 7.9.2009 and Java 8.
> We are seeing it for all the tables (Filter, Data ..etc ) when checked with
> `lsof` command
> {code:java}
> /nb-1-big-Statistics.db.tmp (deleted)
> /nb-3-big-Statistics.db.tmp (deleted)
> /nb-2-big-Data.db (deleted)
> /nb-2-big-Statistics.db.tmp (deleted)
> /nb-2-big-Index.db (deleted)
> /nb-2-big-Statistics.db.tmp (deleted)
> /nb-2-big-CompressionInfo.db (deleted)
> /nb-2-big-Filter.db (deleted)
> /nb-2-big-Summary.db (deleted)
> /nb-2-big-Digest.crc32 (deleted)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (5a4d43d0 -> 698541f7)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 5a4d43d0 generate docs for cceb2c06 new 698541f7 generate docs for cceb2c06 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (5a4d43d0) \ N -- N -- N refs/heads/asf-staging (698541f7) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/doc/5.0/cassandra/new/index.html | 3 + .../5.0/cassandra/tools/nodetool/bootstrap.html| 6 +- .../doc/5.0/cassandra/tools/nodetool/import.html | 6 +- .../doc/5.0/cassandra/tools/nodetool/nodetool.html | 8 +-- .../5.0/cassandra/tools/nodetool/repair_admin.html | 61 +++-- content/doc/5.1/cassandra/new/index.html | 3 + .../5.1/cassandra/tools/nodetool/bootstrap.html| 6 +- .../doc/5.1/cassandra/tools/nodetool/import.html | 6 +- .../doc/5.1/cassandra/tools/nodetool/nodetool.html | 8 +-- .../5.1/cassandra/tools/nodetool/repair_admin.html | 61 +++-- content/doc/trunk/cassandra/new/index.html | 3 + .../trunk/cassandra/tools/nodetool/bootstrap.html | 6 +- .../doc/trunk/cassandra/tools/nodetool/import.html | 6 +- .../trunk/cassandra/tools/nodetool/nodetool.html | 8 +-- .../cassandra/tools/nodetool/repair_admin.html | 61 +++-- site-ui/build/ui-bundle.zip| Bin 4852435 -> 4852435 bytes 16 files changed, 132 insertions(+), 120 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18773) Compactions are slow
[
https://issues.apache.org/jira/browse/CASSANDRA-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755500#comment-17755500
]
Stefan Miklosovic edited comment on CASSANDRA-18773 at 8/17/23 11:15 AM:
-
Thank you [~cam1982] for finally putting this together. We were looking into
this with Cam for some time and the algorithm dealing with ManyToOne is
creating object per iterator / per sstable and it is just super slow on
clusters with thousands of sstables.
I might add that this is not just a lab-like exercise but we have customers for
who we try to do major compactions and we are unable to do that in a reasonable
time due to its extreme slowness.
was (Author: smiklosovic):
Thank you [~cam1982] for finally putting this together. We were looking into
this with Cam for some time and the algorithm dealing with ManyToOne is
creating object per iterator / per sstable and it is just super slow on
clusters with thousands of sstables.
> Compactions are slow
>
>
> Key: CASSANDRA-18773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18773
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction
>Reporter: Cameron Zemek
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Attachments: stress.yaml
>
>
> I have noticed that compactions involving a lot of sstables are very slow
> (for example major compactions). I have attached a cassandra stress profile
> that can generate such a dataset under ccm. In my local test I have 2567
> sstables at 4Mb each.
> I added code to track wall clock time of various parts of the code. One
> problematic part is ManyToOne constructor. Tracing through the code for every
> partition creating a ManyToOne for all the sstable iterators for each
> partition. In my local test get a measy 60Kb/sec read speed, and bottlenecked
> on single core CPU (since this code is single threaded) with it spending 85%
> of the wall clock time in ManyToOne constructor.
> As another datapoint to show its the merge iterator part of the code using
> the cfstats from [https://github.com/instaclustr/cassandra-sstable-tools/]
> which reads all the sstables but does no merging gets 26Mb/sec read speed.
> Tracking back from ManyToOne call I see this in
> UnfilteredPartitionIterators::merge
> {code:java}
> for (int i = 0; i < toMerge.size(); i++)
> {
> if (toMerge.get(i) == null)
> {
> if (null == empty)
> empty = EmptyIterators.unfilteredRow(metadata,
> partitionKey, isReverseOrder);
> toMerge.set(i, empty);
> }
> }
> {code}
> Not sure what purpose of creating these empty rows are. But on a whim I
> removed all these empty iterators before passing to ManyToOne and then all
> the wall clock time shifted to CompactionIterator::hasNext() and read speed
> increased to 1.5Mb/s.
> So there are further bottlenecks in this code path it seems, but the first is
> this ManyToOne and having to build it for every partition read.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18773) Compactions are slow
[
https://issues.apache.org/jira/browse/CASSANDRA-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755500#comment-17755500
]
Stefan Miklosovic commented on CASSANDRA-18773:
---
Thank you [~cam1982] for finally putting this together. We were looking into
this with Cam for some time and the algorithm dealing with ManyToOne is
creating object per iterator / per sstable and it is just super slow on
clusters with thousands of sstables.
> Compactions are slow
>
>
> Key: CASSANDRA-18773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18773
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction
>Reporter: Cameron Zemek
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Attachments: stress.yaml
>
>
> I have noticed that compactions involving a lot of sstables are very slow
> (for example major compactions). I have attached a cassandra stress profile
> that can generate such a dataset under ccm. In my local test I have 2567
> sstables at 4Mb each.
> I added code to track wall clock time of various parts of the code. One
> problematic part is ManyToOne constructor. Tracing through the code for every
> partition creating a ManyToOne for all the sstable iterators for each
> partition. In my local test get a measy 60Kb/sec read speed, and bottlenecked
> on single core CPU (since this code is single threaded) with it spending 85%
> of the wall clock time in ManyToOne constructor.
> As another datapoint to show its the merge iterator part of the code using
> the cfstats from [https://github.com/instaclustr/cassandra-sstable-tools/]
> which reads all the sstables but does no merging gets 26Mb/sec read speed.
> Tracking back from ManyToOne call I see this in
> UnfilteredPartitionIterators::merge
> {code:java}
> for (int i = 0; i < toMerge.size(); i++)
> {
> if (toMerge.get(i) == null)
> {
> if (null == empty)
> empty = EmptyIterators.unfilteredRow(metadata,
> partitionKey, isReverseOrder);
> toMerge.set(i, empty);
> }
> }
> {code}
> Not sure what purpose of creating these empty rows are. But on a whim I
> removed all these empty iterators before passing to ManyToOne and then all
> the wall clock time shifted to CompactionIterator::hasNext() and read speed
> increased to 1.5Mb/s.
> So there are further bottlenecks in this code path it seems, but the first is
> this ManyToOne and having to build it for every partition read.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18774) pre-commit check for submodule does not consider remotes other than "origin"
[ https://issues.apache.org/jira/browse/CASSANDRA-18774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated CASSANDRA-18774: -- Test and Documentation Plan: manual verification Status: Patch Available (was: Open) https://github.com/apache/cassandra/pull/2606 > pre-commit check for submodule does not consider remotes other than "origin" > > > Key: CASSANDRA-18774 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18774 > Project: Cassandra > Issue Type: Task > Components: Accord, Build >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Low > Time Spent: 10m > Remaining Estimate: 0h > > So when having a feature branch in a private fork, I always get an error -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18774) pre-commit check for submodule does not consider remotes other than "origin"
[ https://issues.apache.org/jira/browse/CASSANDRA-18774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated CASSANDRA-18774: -- Change Category: Code Clarity Complexity: Low Hanging Fruit Priority: Low (was: Normal) Status: Open (was: Triage Needed) > pre-commit check for submodule does not consider remotes other than "origin" > > > Key: CASSANDRA-18774 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18774 > Project: Cassandra > Issue Type: Task > Components: Accord, Build >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Low > Time Spent: 10m > Remaining Estimate: 0h > > So when having a feature branch in a private fork, I always get an error -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18767) tablestats should show speculative retries
[ https://issues.apache.org/jira/browse/CASSANDRA-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755498#comment-17755498 ] Brandon Williams commented on CASSANDRA-18767: -- Nothing related to this. > tablestats should show speculative retries > -- > > Key: CASSANDRA-18767 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18767 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > NodeProbe makes > [provisions|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java#L1921] > for it, but it is never used later. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18767) tablestats should show speculative retries
[ https://issues.apache.org/jira/browse/CASSANDRA-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755496#comment-17755496 ] Maxwell Guo edited comment on CASSANDRA-18767 at 8/17/23 11:01 AM: --- yeah, that really doesn't make any sense. the dtest for 3.11 faild~~~ was (Author: maxwellguo): yeah, that really doesn't make any sense > tablestats should show speculative retries > -- > > Key: CASSANDRA-18767 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18767 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > NodeProbe makes > [provisions|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java#L1921] > for it, but it is never used later. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18767) tablestats should show speculative retries
[ https://issues.apache.org/jira/browse/CASSANDRA-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755496#comment-17755496 ] Maxwell Guo commented on CASSANDRA-18767: - yeah, that really doesn't make any sense > tablestats should show speculative retries > -- > > Key: CASSANDRA-18767 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18767 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > NodeProbe makes > [provisions|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java#L1921] > for it, but it is never used later. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18772) Remove dependency on commons-codec
[ https://issues.apache.org/jira/browse/CASSANDRA-18772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18772: - Reviewers: Brandon Williams > Remove dependency on commons-codec > -- > > Key: CASSANDRA-18772 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18772 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 5.0.x, 5.x > > > The dependency doesn't seem to have any direct usage in our codebase. > The only reference to how it might be used was found in CASSANDRA-12790. > Considering we also deprecate in 5.0 and remove metrics-reporter-config as > part of CASSANDRA-18743, I do not see a reason not to remove this one too. > We should probably take the same approach as CASSANDRA-18743 - deprecate in > 5.0 and remove in 5.1. ML thread is to be opened too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18772) Remove dependency on commons-codec
[ https://issues.apache.org/jira/browse/CASSANDRA-18772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755493#comment-17755493 ] Brandon Williams commented on CASSANDRA-18772: -- CI looks good, I'm +1 assuming we clear the ML. > Remove dependency on commons-codec > -- > > Key: CASSANDRA-18772 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18772 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 5.0.x, 5.x > > > The dependency doesn't seem to have any direct usage in our codebase. > The only reference to how it might be used was found in CASSANDRA-12790. > Considering we also deprecate in 5.0 and remove metrics-reporter-config as > part of CASSANDRA-18743, I do not see a reason not to remove this one too. > We should probably take the same approach as CASSANDRA-18743 - deprecate in > 5.0 and remove in 5.1. ML thread is to be opened too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18773) Compactions are slow
[
https://issues.apache.org/jira/browse/CASSANDRA-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755474#comment-17755474
]
Brandon Williams commented on CASSANDRA-18773:
--
/cc [~marcuse]
> Compactions are slow
>
>
> Key: CASSANDRA-18773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18773
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction
>Reporter: Cameron Zemek
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Attachments: stress.yaml
>
>
> I have noticed that compactions involving a lot of sstables are very slow
> (for example major compactions). I have attached a cassandra stress profile
> that can generate such a dataset under ccm. In my local test I have 2567
> sstables at 4Mb each.
> I added code to track wall clock time of various parts of the code. One
> problematic part is ManyToOne constructor. Tracing through the code for every
> partition creating a ManyToOne for all the sstable iterators for each
> partition. In my local test get a measy 60Kb/sec read speed, and bottlenecked
> on single core CPU (since this code is single threaded) with it spending 85%
> of the wall clock time in ManyToOne constructor.
> As another datapoint to show its the merge iterator part of the code using
> the cfstats from [https://github.com/instaclustr/cassandra-sstable-tools/]
> which reads all the sstables but does no merging gets 26Mb/sec read speed.
> Tracking back from ManyToOne call I see this in
> UnfilteredPartitionIterators::merge
> {code:java}
> for (int i = 0; i < toMerge.size(); i++)
> {
> if (toMerge.get(i) == null)
> {
> if (null == empty)
> empty = EmptyIterators.unfilteredRow(metadata,
> partitionKey, isReverseOrder);
> toMerge.set(i, empty);
> }
> }
> {code}
> Not sure what purpose of creating these empty rows are. But on a whim I
> removed all these empty iterators before passing to ManyToOne and then all
> the wall clock time shifted to CompactionIterator::hasNext() and read speed
> increased to 1.5Mb/s.
> So there are further bottlenecks in this code path it seems, but the first is
> this ManyToOne and having to build it for every partition read.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18773) Compactions are slow
[
https://issues.apache.org/jira/browse/CASSANDRA-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18773:
-
Change Category: Performance
Complexity: Normal
Component/s: Local/Compaction
Fix Version/s: 4.0.x
4.1.x
5.0.x
5.x
Status: Open (was: Triage Needed)
> Compactions are slow
>
>
> Key: CASSANDRA-18773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18773
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction
>Reporter: Cameron Zemek
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Attachments: stress.yaml
>
>
> I have noticed that compactions involving a lot of sstables are very slow
> (for example major compactions). I have attached a cassandra stress profile
> that can generate such a dataset under ccm. In my local test I have 2567
> sstables at 4Mb each.
> I added code to track wall clock time of various parts of the code. One
> problematic part is ManyToOne constructor. Tracing through the code for every
> partition creating a ManyToOne for all the sstable iterators for each
> partition. In my local test get a measy 60Kb/sec read speed, and bottlenecked
> on single core CPU (since this code is single threaded) with it spending 85%
> of the wall clock time in ManyToOne constructor.
> As another datapoint to show its the merge iterator part of the code using
> the cfstats from [https://github.com/instaclustr/cassandra-sstable-tools/]
> which reads all the sstables but does no merging gets 26Mb/sec read speed.
> Tracking back from ManyToOne call I see this in
> UnfilteredPartitionIterators::merge
> {code:java}
> for (int i = 0; i < toMerge.size(); i++)
> {
> if (toMerge.get(i) == null)
> {
> if (null == empty)
> empty = EmptyIterators.unfilteredRow(metadata,
> partitionKey, isReverseOrder);
> toMerge.set(i, empty);
> }
> }
> {code}
> Not sure what purpose of creating these empty rows are. But on a whim I
> removed all these empty iterators before passing to ManyToOne and then all
> the wall clock time shifted to CompactionIterator::hasNext() and read speed
> increased to 1.5Mb/s.
> So there are further bottlenecks in this code path it seems, but the first is
> this ManyToOne and having to build it for every partition read.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17667) Text value containing "/*" interpreted as multiline comment in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-17667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-17667:
-
Status: Open (was: Patch Available)
> Text value containing "/*" interpreted as multiline comment in cqlsh
>
>
> Key: CASSANDRA-17667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17667
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: ANOOP THOMAS
>Assignee: Vineet Gali
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
>
> I use CQLSH command line utility to load some DDLs. The version of utility I
> use is this:
> {noformat}
> [cqlsh 6.0.0 | Cassandra 4.0.0.47 | CQL spec 3.4.5 | Native protocol
> v5]{noformat}
> Command that loads DDL.cql:
> {noformat}
> cqlsh -u username -p password cassandra.example.com 65503 --ssl -f DDL.cql
> {noformat}
> I have a line in CQL script that breaks the syntax.
> {noformat}
> INSERT into tablename (key,columnname1,columnname2) VALUES
> ('keyName','value1','/value2/*/value3');{noformat}
> {{/*}} here is interpreted as start of multi-line comment. It used to work on
> older versions of cqlsh. The error I see looks like this:
> {noformat}
> SyntaxException: line 4:2 mismatched input 'Update' expecting ')'
> (...,'value1','/value2INSERT into tablename(INSERT into tablename
> (key,columnname1,columnname2)) VALUES ('[Update]-...) SyntaxException: line
> 1:0 no viable alternative at input '(' ([(]...)
> {noformat}
> Same behavior while running in interactive mode too. {{/*}} inside a CQL
> statement should not be interpreted as start of multi-line comment.
> With schema:
> {code:java}
> CREATE TABLE tablename ( key text primary key, columnname1 text, columnname2
> text);{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18228) Reorganize Apache C* docs in trunk branch to match the IA
[ https://issues.apache.org/jira/browse/CASSANDRA-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755472#comment-17755472 ] Andres de la Peña commented on CASSANDRA-18228: --- [~polandll] [~mck] this seems to have deleted [the section about dynamic data masking (CEP-20)|https://github.com/apache/cassandra/blob/63ab8e09286e4b2076695f62e08a7f39cb6860d1/doc/modules/cassandra/pages/cql/dynamic_data_masking.adoc]. > Reorganize Apache C* docs in trunk branch to match the IA > - > > Key: CASSANDRA-18228 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18228 > Project: Cassandra > Issue Type: New Feature > Components: Documentation >Reporter: Lorina Poland >Assignee: Lorina Poland >Priority: Normal > Fix For: 5.0 > > > An agreed-upon Information Architecture will be used to reorganize the C* 5.0 > docs. > IA doc: > https://docs.google.com/document/d/1A96K73vj9MbJoD7wJNgIKWrOkLq-ZL2cNZAEXSWrciY/edit?usp=sharing -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18774) pre-commit check for submodule does not consider remotes other than "origin"
Jacek Lewandowski created CASSANDRA-18774: - Summary: pre-commit check for submodule does not consider remotes other than "origin" Key: CASSANDRA-18774 URL: https://issues.apache.org/jira/browse/CASSANDRA-18774 Project: Cassandra Issue Type: Task Components: Accord, Build Reporter: Jacek Lewandowski Assignee: Jacek Lewandowski So when having a feature branch in a private fork, I always get an error -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18767) tablestats should show speculative retries
[ https://issues.apache.org/jira/browse/CASSANDRA-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755452#comment-17755452 ] Brandon Williams commented on CASSANDRA-18767: -- bq. the test case for speculative retries are all default value of 0 All of those tests are just regurgitating what is in the template, which happens to be zero. I can make it something else but the change is purely cosmetic. > tablestats should show speculative retries > -- > > Key: CASSANDRA-18767 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18767 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > NodeProbe makes > [provisions|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java#L1921] > for it, but it is never used later. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18729) Remove unnecessary Netty dependencies after upgrade to 4.1.96
[ https://issues.apache.org/jira/browse/CASSANDRA-18729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18729: -- Fix Version/s: 5.0 (was: 5.x) Source Control Link: https://github.com/apache/cassandra/commit/aa84b19112433f604c9bb9bce4618289b4f6873a Resolution: Fixed Status: Resolved (was: Ready to Commit) > Remove unnecessary Netty dependencies after upgrade to 4.1.96 > - > > Key: CASSANDRA-18729 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18729 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 5.0 > > Time Spent: 50m > Remaining Estimate: 0h > > We introduced a lot of unnecessary dependencies in CASSANDRA-17992 which > should not be there. This ticket is about their removal. > The original patch also got some dependencies wrong. They are 4.1.94 but we > are on 4.1.96. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 555dec50c21a0bed871e186de1429fb5efa8f817 Merge: 64f553bf97 aa84b19112 Author: Stefan Miklosovic AuthorDate: Thu Aug 17 11:25:16 2023 +0200 Merge branch 'cassandra-5.0' into trunk .build/build-resolver.xml | 3 + .build/cassandra-deps-template.xml | 21 +++--- .build/parent-pom-template.xml | 87 +++--- CHANGES.txt| 1 + .../org/apache/cassandra/utils/SigarLibrary.java | 1 + .../distributed/impl/AbstractCluster.java | 1 + 6 files changed, 92 insertions(+), 22 deletions(-) diff --cc CHANGES.txt index 381bb4a227,adf6371512..dac7b11581 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,5 -1,5 +1,6 @@@ -5.0-alpha1 +5.1 +Merged from 5.0: + * Remove unnecessary Netty dependencies after upgrade to version 4.1.96 (CASSANDRA-18729) * Prevent InaccessibleObjectException when the Leak Detector is traversing objects (CASSANDRA-18708) * Remove legacy command line options from cassandra-stress (CASSANDRA-18529) * Remove commitlog_sync_batch_window_in_ms (CASSANDRA-17161) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-5.0 updated (7c355c6a02 -> aa84b19112)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 7c355c6a02 Add CASSANDRA-14227 to 5.0 new features page add aa84b19112 Remove unnecessary Netty dependencies after upgrade to version 4.1.96 No new revisions were added by this update. Summary of changes: .build/build-resolver.xml | 3 + .build/cassandra-deps-template.xml | 21 +++--- .build/parent-pom-template.xml | 87 +++--- CHANGES.txt| 1 + .../org/apache/cassandra/utils/SigarLibrary.java | 1 + .../distributed/impl/AbstractCluster.java | 1 + 6 files changed, 92 insertions(+), 22 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
