[jira] [Updated] (CASSANDRA-18472) Docker images can no longer be built due to virtualenv from pip
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-18472:
Reviewers: Berenguer Blasi, Berenguer Blasi (was: Berenguer Blasi)
Berenguer Blasi, Berenguer Blasi
Status: Review In Progress (was: Patch Available)
> Docker images can no longer be built due to virtualenv from pip
> ---
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18472) Docker images can no longer be built due to virtualenv from pip
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-18472:
Status: Ready to Commit (was: Review In Progress)
> Docker images can no longer be built due to virtualenv from pip
> ---
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18472) Docker images can no longer be built due to virtualenv from pip
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719659#comment-17719659
]
Berenguer Blasi commented on CASSANDRA-18472:
-
+1. Throwing out a suggestion of creating a requirements2.7.txt file instead of
hardcoding them in the image, should be easier to change if needed :shrug:
> Docker images can no longer be built due to virtualenv from pip
> ---
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-18180:
-
Test and Documentation Plan:
Patch:
[https://github.com/djatnieks/cassandra/tree/CASSANDRA-18180]
Tested locally with jdk11 and jdk17:
{code:java}
ant test-jvm-dtest-some
-Dtest.name=org.apache.cassandra.distributed.test.SSTableLoaderEncryptionOptionsTest{code}
Status: Patch Available (was: In Progress)
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719644#comment-17719644
]
Ekaterina Dimitrova commented on CASSANDRA-18180:
-
Thank you for the update, [~djatnieks] !
I just pushed CI run with my paid CircleCI account.
I created the configuration by running in the .circleci folder
{code:java}
./generate_11_and_17.sh -p{code}
We can check the results later
[here|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=CASSANDRA-18180]
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (656bdd5bf -> d176f1195)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 656bdd5bf generate docs for 52f24b20 new d176f1195 generate docs for 52f24b20 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (656bdd5bf) \ N -- N -- N refs/heads/asf-staging (d176f1195) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4796900 -> 4796900 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (c951a48fd -> 656bdd5bf)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard c951a48fd generate docs for 52f24b20 new 656bdd5bf generate docs for 52f24b20 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (c951a48fd) \ N -- N -- N refs/heads/asf-staging (656bdd5bf) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4796900 -> 4796900 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18398) CEP-25: Trie-indexed SSTable format
[ https://issues.apache.org/jira/browse/CASSANDRA-18398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719579#comment-17719579 ] Caleb Rackliffe commented on CASSANDRA-18398: - ...and then we need to create the checksumming Jira I guess ;) > CEP-25: Trie-indexed SSTable format > --- > > Key: CASSANDRA-18398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18398 > Project: Cassandra > Issue Type: Improvement > Components: Local/SSTable >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Normal > Fix For: 5.x > > Time Spent: 36h 50m > Remaining Estimate: 0h > > Implementation of Big Trie-Indexed (BTI) SSTable format, per > [CEP-25|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-25%3A+Trie-indexed+SSTable+format]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18366) Test failure: org.apache.cassandra.distributed.test.FailingRepairTest - testFailingMessage[VALIDATION_REQ/parallel/true]
[
https://issues.apache.org/jira/browse/CASSANDRA-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719574#comment-17719574
]
David Capwell commented on CASSANDRA-18366:
---
sorry, just improved my filters and saw I was pinged
Double checking, is this ticket for the fact junit timed out the test, or that
it is flakey in-general?
https://issues.apache.org/jira/browse/CASSANDRA-18366?focusedCommentId=17704821&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17704821
[~smiklosovic] have not looked much at CASSANDRA-18294, but "die" should kill
the jvm, stopping messaging is the wrong behavior... I thought we even test
that this happens in jvm-dtest (we mock out how instances die and make sure
that they do try...).
bq. It fails again, obviously. Basically, when we hit "die" and we stop
transports, for some unknow-yet reason, the code in FailingRepairTest which
waits for this loops forever:
git says I replaced this behavior in CASSANDRA-17116, and that I use isShutdown
and not replicaInstance.killAttempts... I don't know why I made this change, it
actually feels wrong as the test causes a sstable to be corrupted, and the
policy is "die"... so we just stopped validating that the die policy was
properly handled? If I look at
org.apache.cassandra.service.DefaultFSErrorHandler#handleCorruptSSTable this
isn't where we handle die, so not sure why CASSANDRA-18294 changed that... we
handle die in
org.apache.cassandra.utils.JVMStabilityInspector#inspectThrowable(java.lang.Throwable,
java.util.function.Consumer)
{code}
if (DatabaseDescriptor.getDiskFailurePolicy() == Config.DiskFailurePolicy.die)
if (t instanceof FSError || t instanceof CorruptSSTableException)
isUnstable = true;
// Check for file handle exhaustion
if (t instanceof FileNotFoundException || t instanceof
FileSystemException || t instanceof SocketException)
if (t.getMessage() != null && t.getMessage().contains("Too many
open files"))
isUnstable = true;
if (isUnstable)
{
if (!StorageService.instance.isDaemonSetupCompleted())
FileUtils.handleStartupFSError(t);
killer.killCurrentJVM(t);
}
{code}
Now, if the issue is not that the tests are failing and just that we keep
timing out... I sadly feel the best solution is to do what we always do
(sigh)... split the test cross different class files
{code}
@Parameters(name = "{0}/{1}/{2}")
public static Collection messages()
{
List tests = new ArrayList<>();
for (RepairParallelism parallelism : RepairParallelism.values())
{
for (Boolean withTracing : Arrays.asList(Boolean.TRUE,
Boolean.FALSE))
{
tests.add(new Object[]{ Verb.VALIDATION_REQ, parallelism,
withTracing, failingReaders(Verb.VALIDATION_REQ, parallelism, withTracing) });
}
}
return tests;
}
{code}
This is 3 * 2 = 6 tests... we prob want to split it at the RepairParallelism
level so we have 3 top level tests that do 2 different cases (w/, and w/o
tracing)
> Test failure: org.apache.cassandra.distributed.test.FailingRepairTest -
> testFailingMessage[VALIDATION_REQ/parallel/true]
>
>
> Key: CASSANDRA-18366
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18366
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x
>
>
> First seen
> [here|https://app.circleci.com/pipelines/github/driftx/cassandra/928/workflows/f4e93a72-d4aa-47a2-996f-aa3fb018d848/jobs/16206]
> this test times out for me consistently on both j8 and j11 where 4.1 and
> trunk do not.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18399) Add simple helper script for commiting a change to multiple branches
[
https://issues.apache.org/jira/browse/CASSANDRA-18399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719571#comment-17719571
]
David Capwell commented on CASSANDRA-18399:
---
Thanks [~jmckenzie] for the ping... I just saw this (improved my filters to
better see @s)...
Cool to see this work, and would be great for me to get rid of my own commit
scripts... [~yifanc]/[~jonmeredith]/[~maedhroz] as I know you all use my
script...
I sadly don't have time to review and help with this work, but this is what I
have myself, so merging with this work over time is desirable for me
{code}
[dcapwell ~/src/github/apache/cassandra-commit-scripts] (master) $ ls -1
README.md
ci-test
ci-test-loop
circleci-approval.awk
circleci-enable.py
circleci-fix-parallelism.awk
circleci-verify.py
cleanup-remotes-and-branches.sh
commit-message.sh
dtest-jars.sh
env.sh.template
functions.sh
generate-empty-commit.sh
git-apply-branch
git-apply-commit
git-apply-pr
higher
jira-assignee.sh
jira-authors.sh
jira-issue.sh
jira-reviewers.sh
jira-summary.sh
merge-commit.sh
multibranch-commit.sh
notes
oss-author-lookup.py
oss-diff.sh
oss-pr-ci.sh
template-start-commit-of-mine.sh
template-start-commit.sh
trigger-jenkins-build
update-jvm-upgrade-test-branches.awk
use-python-dtest-repo.sh
{code}
The main thing I have people would care about are
{code}
multibranch-commit.sh -- commit script that knows how to merge cross multiple
branches and repos, it enforces best practices (as defined by me)
circleci-enable.py -- script to manipulate circle ci to make it faster and run
what we care about (by default enables all tests and makes sure _repeat is
enabled... but you can filter for only the domain you care for)
{code}
Glad to see this work and looking forward to it evolving over time!
> Add simple helper script for commiting a change to multiple branches
>
>
> Key: CASSANDRA-18399
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18399
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Low
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> This is just a simple additional script which committers can use to help them
> merge commits, especially if the change applies to multiple Cassandra
> versions.
> For example, for CASSANDRA-18153, which applies to 3.0..trunk, it generates
> the following commands, which then can be run manually:
> {noformat}
> git fetch origin
> git fetch jacek-lewandowski
> # jacek-lewandowski/CASSANDRA-18153-3.0 -> cassandra-3.0
> #
>
> git switch cassandra-3.0
> git reset --hard origin/cassandra-3.0
> git cherry-pick e7e9b42559 # e7e9b42559 Save host id to system.local and
> flush immediately after startup
> git cherry-pick -n 99e96a4f95 && git commit -a --amend --no-edit # 99e96a4f95
> fixes
> git cherry-pick -n c63e3f29f1 && git commit -a --amend --no-edit # c63e3f29f1
> DO NOT MERGE - CircleCI config
> grep 'CASSANDRA-18153' CHANGES.txt || sed -E -i '/^[0-9]+\.[0-9]+/{s/.*/&\n\
> * Save host id to system.local and flush immediately after startup
> (CASSANDRA-18153)/;:a;n;ba}' CHANGES.txt
> git diff CHANGES.txt
> git add CHANGES.txt
> git commit --amend --no-edit
> (git diff origin/cassandra-3.0..HEAD -- .circleci/ | git apply -R --index) &&
> git commit -a --amend --no-edit # Remove all changes in .circleci directory
> if you need to
> git diff --name-only origin/cassandra-3.0..HEAD # print a list of all changes
> files
> # jacek-lewandowski/CASSANDRA-18153-3.11 -> cassandra-3.11
> #
>
> git switch cassandra-3.11
> git reset --hard origin/cassandra-3.11
> git merge -s ours --log --no-edit cassandra-3.0
> git cherry-pick -n 79e7b90c8f && git commit -a --amend --no-edit # 79e7b90c8f
> Save host id to system.local and flush immediately after startup
> git cherry-pick -n bdca4c384b && git commit -a --amend --no-edit # bdca4c384b
> fixes
> git cherry-pick -n 45078e6793 && git commit -a --amend --no-edit # 45078e6793
> DO NOT MERGE - CircleCI config
> grep 'CASSANDRA-18153' CHANGES.txt || sed -E -i '/^Merged from 3.0/{s/.*/&\n\
> * Save host id to system.local and flush immediately after startup
> (CASSANDRA-18153)/;:a;n;ba}' CHANGES.txt
> git diff CHANGES.txt
> git add CHANGES.txt
> git commit --amend --no-edit
> (git diff origin/cassandra-3.11..HEAD -- .circleci/ | git apply -R --index)
> && git commit -a --amend --no-edit # Remove all changes in .circleci
> directory if you need to
> git diff --name-only origin/cassandra-3.11..HEAD # print a list of
[jira] [Assigned] (CASSANDRA-17580) Clients using JMX are unable to handle non-standard java types but we leak this into our Exceptions
[ https://issues.apache.org/jira/browse/CASSANDRA-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell reassigned CASSANDRA-17580: - Assignee: (was: David Capwell) > Clients using JMX are unable to handle non-standard java types but we leak > this into our Exceptions > --- > > Key: CASSANDRA-17580 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17580 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Observability, Observability/JMX >Reporter: David Capwell >Priority: Normal > Fix For: 5.x > > Time Spent: 40m > Remaining Estimate: 0h > > This is an extension of CASSANDRA-17527. > When we throw in JMX the exception gets serialized and sent over the wire, > and if the client doesn’t have the same class path it fails and the exception > is lost and replaced with a ClassNotFoundException. This is bad as the user > has no idea what the issue is so unable to resolve it. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17580) Clients using JMX are unable to handle non-standard java types but we leak this into our Exceptions
[ https://issues.apache.org/jira/browse/CASSANDRA-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-17580: -- Status: In Progress (was: Changes Suggested) > Clients using JMX are unable to handle non-standard java types but we leak > this into our Exceptions > --- > > Key: CASSANDRA-17580 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17580 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Observability, Observability/JMX >Reporter: David Capwell >Priority: Normal > Fix For: 5.x > > Time Spent: 40m > Remaining Estimate: 0h > > This is an extension of CASSANDRA-17527. > When we throw in JMX the exception gets serialized and sent over the wire, > and if the client doesn’t have the same class path it fails and the exception > is lost and replaced with a ClassNotFoundException. This is bad as the user > has no idea what the issue is so unable to resolve it. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17580) Clients using JMX are unable to handle non-standard java types but we leak this into our Exceptions
[ https://issues.apache.org/jira/browse/CASSANDRA-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719568#comment-17719568 ] David Capwell commented on CASSANDRA-17580: --- removed myself from assigned as I stopped working on this > Clients using JMX are unable to handle non-standard java types but we leak > this into our Exceptions > --- > > Key: CASSANDRA-17580 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17580 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Observability, Observability/JMX >Reporter: David Capwell >Priority: Normal > Fix For: 5.x > > Time Spent: 40m > Remaining Estimate: 0h > > This is an extension of CASSANDRA-17527. > When we throw in JMX the exception gets serialized and sent over the wire, > and if the client doesn’t have the same class path it fails and the exception > is lost and replaced with a ClassNotFoundException. This is bad as the user > has no idea what the issue is so unable to resolve it. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (aa700d8a7 -> c951a48fd)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard aa700d8a7 generate docs for 52f24b20 new c951a48fd generate docs for 52f24b20 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (aa700d8a7) \ N -- N -- N refs/heads/asf-staging (c951a48fd) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4796900 -> 4796900 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18180) bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719564#comment-17719564
]
dan jatnieks commented on CASSANDRA-18180:
--
I created a patch that implements {{sun.nio.ch.DirectBuffer}} in
{{BufferPool$Chunk}} and {{Ref}} - doing this resolves the
{{ClassCastException}} coming from {{GaloisCounterMode.overlapDetection}}.
Since {{DirectBuffer}} is not publicly exposed, to make compilation work for
JDK11/17 it's necessary to add exports for {{java.base/jdk.internal.ref}} and
{{java.base/sun.nio.ch}}.
There is also a change to how {{TestNameCheckTask}} is invoked by
{{build.xml}}. Since {{TestNameCheckTask}} uses {{Reflections}} to get the
methods annotated with {{@Test}}, and that involves loading classes with the
ClassLoader it also needs to pass jvm args for jdk11/17 to avoid
{{IllegalAccessError}} to {{sun.nio.ch.DirectBuffer}} (not exported by
{{java.base}}). To accomplish this, the {{checktestnameshelper}} ant task in
{{build.xml}} is replaced with a java task, passing the necessary jvm args.
Note: these changes are not compatible with JDK8.
||Branch||CI||
|[trunk|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18180]|[java
11|https://app.circleci.com/pipelines/github/djatnieks/cassandra/3/workflows/7ba8f619-835e-4464-b5a0-6c828352e3d0]
[java
17|https://app.circleci.com/pipelines/github/djatnieks/cassandra/3/workflows/86cc2e85-e718-4d09-aa55-fbec0c399733]|
> bulkLoaderSuccessfullyStreamsOverSsl fails with ClassCastException on JDK17
> ---
>
> Key: CASSANDRA-18180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18180
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: dan jatnieks
>Priority: Normal
>
> While working on CASSANDRA-17992 we hit:
> {code:java}
> java.lang.ClassCastException: class
> org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class
> sun.nio.ch.DirectBuffer (org.apache.cassandra.utils.memory.BufferPool$Chunk
> is in unnamed module of loader 'app'; sun.nio.ch.DirectBuffer is in module
> java.base of loader 'bootstrap')\n\tat
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMEngine.overlapDetection(GaloisCounterMode.java:865)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode$GCMDecrypt.doFinal(GaloisCounterMode.java:1502)\n\tat
>
> java.base/com.sun.crypto.provider.GaloisCounterMode.engineDoFinal(GaloisCounterMode.java:447)\n\tat
>
> {code}
> -The issue is exposed with JDK 17, trunk; if interested, ping- [~e.dimitrova]
> -for current branch as there is no feature branch at the moment- we can
> build and start from trunk with JDK17 already. Circle CI can be run for JDK17
> too. For more information how to do that - .circleci/readme.md
> CC [~benedict]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18436) Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally failing with JDK17
[
https://issues.apache.org/jira/browse/CASSANDRA-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719555#comment-17719555
]
dan jatnieks commented on CASSANDRA-18436:
--
I can reproduce this locally with IntelliJ IDE, but not on the command line. I
added some error checking to the {{Process}} result and was able to understand
that while my IDE is using JDK17, the default java version on my system is
still java 8. Example of test result in IntelliJ
{code:java}
java.lang.AssertionError: Error: A JNI error has occurred, please check your
installation and try again
Exception in thread "main" java.lang.UnsupportedClassVersionError:
org/apache/cassandra/tools/SSTableExport has been compiled by a more recent
version of the Java Runtime (class file version 61.0), this version of the Java
Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:601)
Expected :0
Actual :1 {code}
Maybe something like this would be useful to add?
||Branch||CI||
|[trunk|https://github.com/djatnieks/cassandra/tree/CASSANDRA-18436-trunk]|[java17|https://app.circleci.com/pipelines/github/djatnieks/cassandra/5/workflows/7678e370-43d6-4a94-a50b-ed30c72257f0]|
> Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally
> failing with JDK17
> -
>
> Key: CASSANDRA-18436
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18436
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 5.x
>
>
>
> All of them failed with the below stack trace for the same assertion failing:
> {code:java}
> junit.framework.AssertionFailedError: at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:90) at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)
> at
> org.apache.cassandra.cql3.EmptyValuesTest.testEmptyDecimal(EmptyValuesTest.java:192)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
> Unfortunately I do not have a link to the CI run as this was seen last in
> private infra and not in CircleCI. Maybe we want to check with the
> multiplexer for flakiness.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18398) CEP-25: Trie-indexed SSTable format
[ https://issues.apache.org/jira/browse/CASSANDRA-18398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719538#comment-17719538 ] Caleb Rackliffe edited comment on CASSANDRA-18398 at 5/4/23 9:48 PM: - Alright, I've left one more round of nits, but at this point, I think I've provided all the feedback I can, and I'll +1 once all those little things have been resolved. Given the successful Harry runs I was able to do earlier, the only other thing I suppose we'd be waiting for is clean CI. I'm really excited to see this move forward! was (Author: maedhroz): Alright, I've left one more round of nits, but at this point, I think I'd provided all the feedback I can, and I'll +1 once all those little things have been resolved. Given the successful Harry runs I was able to do earlier, the only other thing I suppose we'd be waiting for is clean CI. I'm really excited to see this move forward! > CEP-25: Trie-indexed SSTable format > --- > > Key: CASSANDRA-18398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18398 > Project: Cassandra > Issue Type: Improvement > Components: Local/SSTable >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Normal > Fix For: 5.x > > Time Spent: 36h 50m > Remaining Estimate: 0h > > Implementation of Big Trie-Indexed (BTI) SSTable format, per > [CEP-25|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-25%3A+Trie-indexed+SSTable+format]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18398) CEP-25: Trie-indexed SSTable format
[ https://issues.apache.org/jira/browse/CASSANDRA-18398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719538#comment-17719538 ] Caleb Rackliffe commented on CASSANDRA-18398: - Alright, I've left one more round of nits, but at this point, I think I'd provided all the feedback I can, and I'll +1 once all those little things have been resolved. Given the successful Harry runs I was able to do earlier, the only other thing I suppose we'd be waiting for is clean CI. I'm really excited to see this move forward! > CEP-25: Trie-indexed SSTable format > --- > > Key: CASSANDRA-18398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18398 > Project: Cassandra > Issue Type: Improvement > Components: Local/SSTable >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Normal > Fix For: 5.x > > Time Spent: 36h 50m > Remaining Estimate: 0h > > Implementation of Big Trie-Indexed (BTI) SSTable format, per > [CEP-25|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-25%3A+Trie-indexed+SSTable+format]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18398) CEP-25: Trie-indexed SSTable format
[ https://issues.apache.org/jira/browse/CASSANDRA-18398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719537#comment-17719537 ] Caleb Rackliffe commented on CASSANDRA-18398: - bq. it sounds like SAI will have similar issues, so it may be good to flesh out how to detect this for both, but primary index is more important than SAI so if w/e the solution is is localized to primary index If we do something at the trie level, it seems like it should reach to the trie-based terms dictionary SAI uses for string data without any additional effort. > CEP-25: Trie-indexed SSTable format > --- > > Key: CASSANDRA-18398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18398 > Project: Cassandra > Issue Type: Improvement > Components: Local/SSTable >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Normal > Fix For: 5.x > > Time Spent: 36h 50m > Remaining Estimate: 0h > > Implementation of Big Trie-Indexed (BTI) SSTable format, per > [CEP-25|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-25%3A+Trie-indexed+SSTable+format]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16718) Changing listen_address with prefer_local may lead to issues
[ https://issues.apache.org/jira/browse/CASSANDRA-16718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719534#comment-17719534 ] Matt Fleming commented on CASSANDRA-16718: -- Patch looks good to me. nb +1 > Changing listen_address with prefer_local may lead to issues > > > Key: CASSANDRA-16718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16718 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Jan Karlsson >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.x > > > Many container based solution function by assigning new listen_addresses when > nodes are stopped. Changing the listen_address is usually as simple as > turning off the node and changing the yaml file. > However, if prefer_local is enabled, I observed that nodes were unable to > join the cluster and fail with 'Unable to gossip with any seeds'. > Trace shows that the changing node will try to communicate with the existing > node but the response is never received. I assume it is because the existing > node attempts to communicate with the local address during the shadow round. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17918) DESCRIBE output does not quote column names using reserved keywords
[
https://issues.apache.org/jira/browse/CASSANDRA-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719367#comment-17719367
]
Stefan Miklosovic edited comment on CASSANDRA-17918 at 5/4/23 9:24 PM:
---
The problem with tests is that the patch introduces this result of aggregate
description, for example it would describe an aggregate like this:
{code:java}
CREATE AGGREGATE cqlshtests_dligxmowmx.aggmax(int)
SFUNC cqlshtests_dligxmowmx."fmax"
STYPE int; {code}
This does not work because the the value of SFUNC is in form "keyspace.name"
and it does not accept this. There can not be ".". It just expects name of the
function.
[https://github.com/apache/cassandra/blob/trunk/src/antlr/Parser.g#L671]
[https://github.com/apache/cassandra/blob/trunk/src/antlr/Parser.g#L1567]
So the original author introduced something to CQL but parser was not modified.
I do not know how to fix this. I think we should just leave it out as it was
and we should just quote the function name. If we are creating an aggregate on
some keyspace, I do not think that SFUNC is going to ever be something else
from a function in that keyspace. So the further distinction what keyspace that
SFUNC is from does not make a lot of sense.
my PR for trunk is here which builds on top of the original work:
[https://github.com/apache/cassandra/pull/2308]
https://app.circleci.com/pipelines/github/instaclustr/cassandra/2177/workflows/36b2e8b5-1ea8-4579-bc4a-5dca6db4e07f
was (Author: smiklosovic):
The problem with tests is that the patch introduces this result of aggregate
description, for example it would describe an aggregate like this:
{code:java}
CREATE AGGREGATE cqlshtests_dligxmowmx.aggmax(int)
SFUNC cqlshtests_dligxmowmx."fmax"
STYPE int; {code}
This does not work because the the value of SFUNC is in form "keyspace.name"
and it does not accept this. There can not be ".". It just expects name of the
function.
[https://github.com/apache/cassandra/blob/trunk/src/antlr/Parser.g#L671]
[https://github.com/apache/cassandra/blob/trunk/src/antlr/Parser.g#L1567]
So the original author introduced something to CQL but parser was not modified.
I do not know how to fix this. I think we should just leave it out as it was
and we should just quote the function name. If we are creating an aggregate on
some keyspace, I do not think that SFUNC is going to ever be something else
from a function in that keyspace. So the further distinction what keyspace that
SFUNC is from does not make a lot of sense.
my PR for trunk is here which builds on top of the original work:
https://github.com/apache/cassandra/pull/2308
> DESCRIBE output does not quote column names using reserved keywords
> ---
>
> Key: CASSANDRA-17918
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17918
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
>
> The DESCRIBE output of the column names that using reserved keywords are not
> quoted for UDTs. The following test reproduces. Reading the code, it looks
> like that the such columns names are not quoted in materialized view, UDF and
> user defined aggregation.
> The impact of the bug is that schema described cannot be imported due to the
> usage of reserved keywords as column names.
>
> {code:java}
> @Test
> public void testUsingReservedInCreateType() throws Throwable
> {
> String type = createType(KEYSPACE_PER_TEST, "CREATE TYPE %s
> (\"token\" text, \"desc\" text);");
> assertRowsNet(executeDescribeNet(KEYSPACE_PER_TEST, "DESCRIBE TYPE " + type),
> row(KEYSPACE_PER_TEST, "type", type, "CREATE TYPE " +
> KEYSPACE_PER_TEST + "." + type + " (\n" +
> " \"token\" text,\n" +
> " \"desc\" text\n" +
> ");"));
> } {code}
> +Additional information for newcomers:+
> * Unit tests for DESCRIBE statements are in {{DescribeStatementTest}}
> * The statement implementation is in {{DescribeStatement and fetch the
> create statement from the different schema element using
> SchemaElement.toCqlString}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18344) Store PreAccept, Accept, Commit, and Apply in a durable log before processing by CommandStores
[ https://issues.apache.org/jira/browse/CASSANDRA-18344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719532#comment-17719532 ] David Capwell commented on CASSANDRA-18344: --- +1 > Store PreAccept, Accept, Commit, and Apply in a durable log before processing > by CommandStores > -- > > Key: CASSANDRA-18344 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18344 > Project: Cassandra > Issue Type: New Feature > Components: Accord >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 5.x > > Time Spent: 17h > Remaining Estimate: 0h > > Write PreAccept, Accept, Commit, and Apply messages durably to a journal. > CommandStore will then processes the messages in write order. > > This is a prerequisite JIRA for several incoming improvements, such as > reducing the data we store in system tables, making command stores and > progress log persistent, simplifying caching. > The journal supports invalidating individual entries by id and sets of > owners, and looking up records by id, both needed for near-future Accord work. > > Several complete/near-complete parts of the generic journal implementation > have been temporarily taken out until we need them (soon, once > bootstrap/state reclamation are implemented). > These include invalidation support (in memory and on-disk implementations), > and segment compaction (not needed without invalidations). > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18441) Improvements to SSTable format configuration
[ https://issues.apache.org/jira/browse/CASSANDRA-18441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719531#comment-17719531 ] David Capwell commented on CASSANDRA-18441: --- Reviewed, very small feedback but overall LGTM. Once you fix cassandra.yml I am +1 > Improvements to SSTable format configuration > > > Key: CASSANDRA-18441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18441 > Project: Cassandra > Issue Type: Improvement > Components: Local/SSTable >Reporter: Branimir Lambov >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 5.x > > > CEP-17 and CASSANDRA-17056 abstracted some interfaces for SSTable format > implementations and defined a method of plugging in specific configurations. > This method is brittle and asks users to specify format identifiers whose > configuration does not provide value but can be the source of conflicts and > problems. On the other hand it makes important choices non-obvious, as the > selection of format to write is given by the order of configured interfaces. > An improved specification mechanism needs to be put in place before Cassandra > 5 is released. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18485) CEP-15: (C*) Enhance in-memory FileSystem to work with mmap and support tests to add custom logic
[ https://issues.apache.org/jira/browse/CASSANDRA-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719527#comment-17719527 ] David Capwell commented on CASSANDRA-18485: --- bq. You mainly change the code to have a pluggable FS This was done already, in Cassandra we block using Java File directly and force you to use Cassandra java, this was done so simulator could use an in-memory file system (google's jimfs). This patch adds a new listable file system that lets you pre/post hooks on top of a file system, so you can add your own faults. Then I change stuff like CQLTester to make it cleaner for tests to do this. bq. Asking bc my simulator knowledge is 0 I want to make sure the above is the surface area of the review. This patch doesn't really touch simulator, follow up work would use this as that work is to add disk faults in simulation. Having a file system that you can add faults would be useful outside of simulation, which is why I tried to get this working with CQLTester > CEP-15: (C*) Enhance in-memory FileSystem to work with mmap and support tests > to add custom logic > - > > Key: CASSANDRA-18485 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18485 > Project: Cassandra > Issue Type: Improvement > Components: Accord, Test/dtest/java, Test/unit >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 5.x > > Time Spent: 1h 50m > Remaining Estimate: 0h > > The Simulator currently uses JimFS for its FileSystem, but this lacks any way > to add custom logic when file operations are performed; it also lacks mmap > support which requires all disk access logic to provide non-mmap solutions as > well. > As part of the Simulator work, testing disk corruption will require resolving > both these issues, so will need a new FileSystem to rule them all… > This ticket is to define the new FileSystem and add the integration to > Simulator, JVM-Dtest, and Unit tests, but does not directly add the fault > injections that Simulator will be doing, that will be follow up work. > Goals: > * FileSystem that works for unit, jvm-dtest, and simulator tests > * FileSystem that allows tests to intercept file operations -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (dd26f3f02 -> aa700d8a7)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard dd26f3f02 generate docs for 52f24b20 new aa700d8a7 generate docs for 52f24b20 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (dd26f3f02) \ N -- N -- N refs/heads/asf-staging (aa700d8a7) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4796900 -> 4796900 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (664c200d3 -> dd26f3f02)
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a change to branch asf-staging
in repository https://gitbox.apache.org/repos/asf/cassandra-website.git
discard 664c200d3 generate docs for 52f24b20
new dd26f3f02 generate docs for 52f24b20
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (664c200d3)
\
N -- N -- N refs/heads/asf-staging (dd26f3f02)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../cassandra/managing/tools/sstable/index.html| 3 +
.../{sstabledump.html => sstablepartitions.html} | 425 -
.../cassandra/managing/tools/sstable/index.html| 3 +
.../managing/tools/sstable/sstablepartitions.html} | 425 -
content/search-index.js| 2 +-
site-ui/build/ui-bundle.zip| Bin 4796900 -> 4796900
bytes
6 files changed, 317 insertions(+), 541 deletions(-)
copy content/doc/4.2/cassandra/managing/tools/sstable/{sstabledump.html =>
sstablepartitions.html} (73%)
copy content/doc/{4.2/cassandra/managing/tools/sstable/sstabledump.html =>
trunk/cassandra/managing/tools/sstable/sstablepartitions.html} (73%)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16222) CEP-28: Reading and Writing Cassandra Data with Spark Bulk Analytics
[ https://issues.apache.org/jira/browse/CASSANDRA-16222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Rohrer updated CASSANDRA-16222: Description: *Description:* This ticket introduces the Spark-Cassandra Bulk Analytics library and associated Cassandra Sidecar endpoints. Please see [CEP-28|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-28%3A+Reading+and+Writing+Cassandra+Data+with+Spark+Bulk+Analytics] for more details, motivation, and available source code links. was: *Description:* This ticket introduces the Spark-Cassandra Bulk Reader: a Spark library able to read and compact Cassandra raw sstables into SparkSQL along the principles of streaming compaction. The full code is attached as a patch file and will be submitted to a GitHub repo. *Motivation* For analytics or "select *" use cases at scale, the performance is prohibitively expensive to read via the normal CQL read path - either using the Java driver or the Open Source Spark connector. By reading the raw sstables, the bulk reader is able to read with near-zero impact to a production cluster at speeds many orders of magnitudes faster than alternatives. We have seen very good performance results, exporting a 32TB table (~46bn CQL rows) to HDFS in Parquet format in around 1h10m; a 20x reduction compared to previous solutions. By reading from multiple replicas and ‘compacting’ duplicate data together, it can achieve consistency at a user defined level (i.e. ONE, TWO, LOCAL_QUORUM etc). *Overview* This library provides the core code for reading a set of SSTables into SparkSQL through a DataLayer abstraction. The role of the DataLayer is to: * return a SchemaStruct, mapping the Cassandra CQL table schema to the SparkSQL schema. * a list of sstables available for reading. * a method to open an InputStream for any file component of an sstable (e.g. data, compression, summary etc). The PartitionedDataLayer abstraction builds on the DataLayer interface for partitioning Spark workers across a Cassandra token ring - allowing the Spark job to scale linearly - and reading from sufficient Cassandra replicas to achieve a user-specified consistency level. A simple example LocalDataLayer implementation is included for reading from a local file system. Users of the library can build their own implementations to read from wherever they wish e.g. reading from a backup in a cloud storage system, or reading from the snapshot directory of a live Cassandra cluster. At the core, the bulk reader uses the Apache Cassandra CompactionIterator to perform the streaming compaction. As it iterates through the CompactionIterator it deserializes the ByteBuffers, converts into the appropriate SparkSQL data type and pivots each cell into a SparkSQL row. Supporting this library is a robust property-based test framework for writing Cassandra sstables with arbitrary schemas using the CQLSSTableWriter, and reading back through SparkSQL to verify the library achieves both consistency and correctness. Summary: CEP-28: Reading and Writing Cassandra Data with Spark Bulk Analytics (was: Spark-Cassandra Bulk Reader) > CEP-28: Reading and Writing Cassandra Data with Spark Bulk Analytics > > > Key: CASSANDRA-16222 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16222 > Project: Cassandra > Issue Type: New Feature > Components: Tool/external >Reporter: James Berragan >Priority: Normal > Fix For: NA > > Attachments: sparkbulkreader.patch > > > *Description:* > This ticket introduces the Spark-Cassandra Bulk Analytics library and > associated Cassandra Sidecar endpoints. > > Please see > [CEP-28|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-28%3A+Reading+and+Writing+Cassandra+Data+with+Spark+Bulk+Analytics] > for more details, motivation, and available source code links. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18441) Improvements to SSTable format configuration
[ https://issues.apache.org/jira/browse/CASSANDRA-18441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719410#comment-17719410 ] Jacek Lewandowski commented on CASSANDRA-18441: --- I've applied the remaining changes > Improvements to SSTable format configuration > > > Key: CASSANDRA-18441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18441 > Project: Cassandra > Issue Type: Improvement > Components: Local/SSTable >Reporter: Branimir Lambov >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 5.x > > > CEP-17 and CASSANDRA-17056 abstracted some interfaces for SSTable format > implementations and defined a method of plugging in specific configurations. > This method is brittle and asks users to specify format identifiers whose > configuration does not provide value but can be the source of conflicts and > problems. On the other hand it makes important choices non-obvious, as the > selection of format to write is given by the order of configured interfaces. > An improved specification mechanism needs to be put in place before Cassandra > 5 is released. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18467) Update generate-idea-files for J17
[ https://issues.apache.org/jira/browse/CASSANDRA-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719405#comment-17719405 ] Ekaterina Dimitrova commented on CASSANDRA-18467: - Suggestion: the original patch that [~jakubzytka] submitted breaks only Java8 in CI. I suggest we commit it when we drop Java 8. Thoughts? > Update generate-idea-files for J17 > -- > > Key: CASSANDRA-18467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18467 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Jakub Zytka >Priority: Low > Fix For: 5.x > > > There was a discussion in CASSANDRA-18258 how to update generate-idea-files. > The final agreement was to create one target to cover both Java 11 and Java > 17. > It will be good to figure out CASSANDRA-18263 and reshuffle arguments and > tasks based on what we decide to use as gc in testing for both Java 11 and > Java 17. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18497) snakeyaml vulnerability: CVE-2023-2251
[ https://issues.apache.org/jira/browse/CASSANDRA-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18497: - Reviewers: Stefan Miklosovic > snakeyaml vulnerability: CVE-2023-2251 > -- > > Key: CASSANDRA-18497 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18497 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.29, 4.0.10, 4.1.2, 5.0, 3.11.16 > > > This is failing the OWASP scan. > https://nvd.nist.gov/vuln/detail/CVE-2023-2251 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18497) snakeyaml vulnerability: CVE-2023-2251
[ https://issues.apache.org/jira/browse/CASSANDRA-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18497: - Fix Version/s: 3.0.29 4.0.10 4.1.2 5.0 3.11.16 (was: 3.0.x) (was: 3.11.x) (was: 5.x) (was: 4.0.x) (was: 4.1.x) Since Version: NA Source Control Link: https://github.com/apache/cassandra/commit/4a62757624d120a0e493d9d39bcd9bf8282b4f58 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed, thanks. > snakeyaml vulnerability: CVE-2023-2251 > -- > > Key: CASSANDRA-18497 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18497 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.29, 4.0.10, 4.1.2, 5.0, 3.11.16 > > > This is failing the OWASP scan. > https://nvd.nist.gov/vuln/detail/CVE-2023-2251 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18497) snakeyaml vulnerability: CVE-2023-2251
[ https://issues.apache.org/jira/browse/CASSANDRA-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18497: - Status: Review In Progress (was: Needs Committer) > snakeyaml vulnerability: CVE-2023-2251 > -- > > Key: CASSANDRA-18497 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18497 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > > This is failing the OWASP scan. > https://nvd.nist.gov/vuln/detail/CVE-2023-2251 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18497) snakeyaml vulnerability: CVE-2023-2251
[ https://issues.apache.org/jira/browse/CASSANDRA-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18497: - Status: Ready to Commit (was: Review In Progress) > snakeyaml vulnerability: CVE-2023-2251 > -- > > Key: CASSANDRA-18497 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18497 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > > This is failing the OWASP scan. > https://nvd.nist.gov/vuln/detail/CVE-2023-2251 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18500) Add guardrail for partition size
Andres de la Peña created CASSANDRA-18500:
-
Summary: Add guardrail for partition size
Key: CASSANDRA-18500
URL: https://issues.apache.org/jira/browse/CASSANDRA-18500
Project: Cassandra
Issue Type: New Feature
Components: Feature/Guardrails
Reporter: Andres de la Peña
Assignee: Andres de la Peña
Add a guardrail for max partition size, for example:
{code:java}
partition_size_warn_threshold: 50MiB
partition_size_fail_threshold: 100MiB
{code}
Most probably this guardrail would only be checked when writing a new sstable
to disk (fush/compact). Triggering the guardrail on sstable write would emit a
log message and a diagnostic event, but it wouldn't reject the write.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18497) snakeyaml vulnerability: CVE-2023-2251
[ https://issues.apache.org/jira/browse/CASSANDRA-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18497: - Status: Needs Committer (was: Patch Available) > snakeyaml vulnerability: CVE-2023-2251 > -- > > Key: CASSANDRA-18497 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18497 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > > This is failing the OWASP scan. > https://nvd.nist.gov/vuln/detail/CVE-2023-2251 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.1' into trunk
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 42dac9b9500259286d5c9508c08d5e618cf7119c Merge: e72ec4e828 462934a84b Author: Brandon Williams AuthorDate: Thu May 4 10:26:10 2023 -0500 Merge branch 'cassandra-4.1' into trunk .build/dependency-check-suppressions.xml | 1 + CHANGES.txt | 14 ++ 2 files changed, 15 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 02/03: Merge branch 'cassandra-3.11' into cassandra-4.0
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit dba41626669a66dcd78eb7b097988e1472b4a65c Merge: 65c99bfc42 aa73bc468a Author: Brandon Williams AuthorDate: Thu May 4 10:23:47 2023 -0500 Merge branch 'cassandra-3.11' into cassandra-4.0 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.1 updated (82dc54c3a8 -> 462934a84b)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 82dc54c3a8 Merge branch 'cassandra-4.0' into cassandra-4.1 new 4a62757624 Suppress CVE-2023-2251 new aa73bc468a Merge branch 'cassandra-3.0' into cassandra-3.11 new dba4162666 Merge branch 'cassandra-3.11' into cassandra-4.0 new b6b71c598e Merge branch 'cassandra-3.0' into cassandra-3.11 new 682ae0c64c Merge branch 'cassandra-3.11' into cassandra-4.0 new 462934a84b Merge branch 'cassandra-4.0' into cassandra-4.1 The 6 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .build/dependency-check-suppressions.xml | 1 + CHANGES.txt | 14 ++ 2 files changed, 15 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/03: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit aa73bc468aa58805f406c756647384b8608e3c42 Merge: 90d0857d34 4a62757624 Author: Brandon Williams AuthorDate: Thu May 4 10:22:25 2023 -0500 Merge branch 'cassandra-3.0' into cassandra-3.11 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.0' into cassandra-4.1
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 462934a84bac08b87ad8ebfcd2d9e8e6227cfa99 Merge: 82dc54c3a8 682ae0c64c Author: Brandon Williams AuthorDate: Thu May 4 10:26:03 2023 -0500 Merge branch 'cassandra-4.0' into cassandra-4.1 .build/dependency-check-suppressions.xml | 1 + CHANGES.txt | 14 ++ 2 files changed, 15 insertions(+) diff --cc CHANGES.txt index a775ce8ff3,a10b0f67df..c3659802b2 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -10,9 -3,22 +10,23 @@@ Merged from 4.0 * Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest (CASSANDRA-18474) * Deadlock updating sstable metadata if disk boundaries need reloading (CASSANDRA-18443) * Fix nested selection of reversed collections (CASSANDRA-17913) + * Update zstd-jni library to version 1.5.5 (CASSANDRA-18429) Merged from 3.11: * Fix the capital P usage in the CQL parser (CASSANDRA-17919) + Merged from 3.0: + * Suppress CVE-2023-2251 (CASSANDRA-18497) + * Do not remove SSTables when cause of FSReadError is OutOfMemoryError while using best_effort disk failure policy (CASSANDRA-18336) + * Do not remove truncated_at entry in system.local while dropping an index (CASSANDRA-18105) + + 4.0.9 + * Update zstd-jni library to version 1.5.5 (CASSANDRA-18429) + * Backport CASSANDRA-17205 to 4.0 branch - Remove self-reference in SSTableTidier (CASSANDRA-18332) + * Avoid loading the preferred IP for BulkLoader streaming (CASSANDRA-18370) + * Fix BufferPool incorrect memoryInUse when putUnusedPortion is used (CASSANDRA-18311) + * Improve memtable allocator accounting when updating AtomicBTreePartition (CASSANDRA-18125) + * Update zstd-jni to version 1.5.4-1 (CASSANDRA-18259) + * Split and order IDEA workspace template VM_PARAMETERS (CASSANDRA-18242) + Merged from 3.11: * Fix sstable_count metric missing from tablestats json/yaml output (CASSANDRA-18448) * Suppress CVE-2022-45688 (CASSANDRA-18389) * Fix Splitter sometimes creating more splits than requested (CASSANDRA-18013) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated (65c99bfc42 -> 682ae0c64c)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 65c99bfc42 Improve 'Not enough space for compaction' logging messages new 4a62757624 Suppress CVE-2023-2251 new aa73bc468a Merge branch 'cassandra-3.0' into cassandra-3.11 new dba4162666 Merge branch 'cassandra-3.11' into cassandra-4.0 new b6b71c598e Merge branch 'cassandra-3.0' into cassandra-3.11 new 682ae0c64c Merge branch 'cassandra-3.11' into cassandra-4.0 The 5 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .build/dependency-check-suppressions.xml | 1 + CHANGES.txt | 1 + 2 files changed, 2 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (e72ec4e828 -> 42dac9b950)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from e72ec4e828 Add sstablepartitions offline tool to find large partitions in sstables new 4a62757624 Suppress CVE-2023-2251 new aa73bc468a Merge branch 'cassandra-3.0' into cassandra-3.11 new dba4162666 Merge branch 'cassandra-3.11' into cassandra-4.0 new b6b71c598e Merge branch 'cassandra-3.0' into cassandra-3.11 new 682ae0c64c Merge branch 'cassandra-3.11' into cassandra-4.0 new 462934a84b Merge branch 'cassandra-4.0' into cassandra-4.1 new 42dac9b950 Merge branch 'cassandra-4.1' into trunk The 7 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .build/dependency-check-suppressions.xml | 1 + CHANGES.txt | 14 ++ 2 files changed, 15 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 03/03: Merge branch 'cassandra-3.11' into cassandra-4.0
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 682ae0c64c9159c6c884252820ce1236fde39e17 Merge: dba4162666 b6b71c598e Author: Brandon Williams AuthorDate: Thu May 4 10:24:20 2023 -0500 Merge branch 'cassandra-3.11' into cassandra-4.0 .build/dependency-check-suppressions.xml | 1 + CHANGES.txt | 1 + 2 files changed, 2 insertions(+) diff --cc .build/dependency-check-suppressions.xml index f203c2757f,98efddce73..f8aa4b5fb7 --- a/.build/dependency-check-suppressions.xml +++ b/.build/dependency-check-suppressions.xml @@@ -21,17 -21,10 +21,18 @@@ --> https://jeremylong.github.io/DependencyCheck/dependency-suppression.1.3.xsd";> - + +^pkg:maven/com\.datastax\.cassandra/cassandra\-driver\-core@.*$ +CVE-2018-8016 +CVE-2019-2684 +CVE-2020-13946 +CVE-2020-17516 +CVE-2021-44521 + + + ^pkg:maven/org\.yaml/snakeyaml@.*$ + CVE-2023-2251 -CVE-2017-18640 CVE-2022-25857 CVE-2022-38749 CVE-2022-38750 diff --cc CHANGES.txt index b3f3cf883a,b132527d5c..a10b0f67df --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,23 -1,9 +1,24 @@@ -3.11.16 +4.0.10 + * Improve 'Not enough space for compaction' logging messages (CASSANDRA-18260) + * Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest (CASSANDRA-18474) + * Deadlock updating sstable metadata if disk boundaries need reloading (CASSANDRA-18443) + * Fix nested selection of reversed collections (CASSANDRA-17913) +Merged from 3.11: + * Fix the capital P usage in the CQL parser (CASSANDRA-17919) Merged from 3.0: + * Suppress CVE-2023-2251 (CASSANDRA-18497) + * Do not remove SSTables when cause of FSReadError is OutOfMemoryError while using best_effort disk failure policy (CASSANDRA-18336) + * Do not remove truncated_at entry in system.local while dropping an index (CASSANDRA-18105) -3.11.15 - * Fix the capital P usage in the CQL parser (CASSANDRA-17919) +4.0.9 + * Update zstd-jni library to version 1.5.5 (CASSANDRA-18429) + * Backport CASSANDRA-17205 to 4.0 branch - Remove self-reference in SSTableTidier (CASSANDRA-18332) + * Avoid loading the preferred IP for BulkLoader streaming (CASSANDRA-18370) + * Fix BufferPool incorrect memoryInUse when putUnusedPortion is used (CASSANDRA-18311) + * Improve memtable allocator accounting when updating AtomicBTreePartition (CASSANDRA-18125) + * Update zstd-jni to version 1.5.4-1 (CASSANDRA-18259) + * Split and order IDEA workspace template VM_PARAMETERS (CASSANDRA-18242) +Merged from 3.11: * Fix sstable_count metric missing from tablestats json/yaml output (CASSANDRA-18448) * Suppress CVE-2022-45688 (CASSANDRA-18389) * Fix Splitter sometimes creating more splits than requested (CASSANDRA-18013) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (90d0857d34 -> b6b71c598e)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 90d0857d34 fix typos in data modeling and getting started docs new 4a62757624 Suppress CVE-2023-2251 new b6b71c598e Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .build/dependency-check-suppressions.xml | 1 + CHANGES.txt | 4 2 files changed, 5 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit b6b71c598eef1a01c52cef0e7fa54c6e24d39095 Merge: 90d0857d34 4a62757624 Author: Brandon Williams AuthorDate: Thu May 4 10:22:25 2023 -0500 Merge branch 'cassandra-3.0' into cassandra-3.11 .build/dependency-check-suppressions.xml | 1 + CHANGES.txt | 4 2 files changed, 5 insertions(+) diff --cc .build/dependency-check-suppressions.xml index 68c97bb777,5a87f57c3f..98efddce73 --- a/.build/dependency-check-suppressions.xml +++ b/.build/dependency-check-suppressions.xml @@@ -23,13 -23,13 +23,14 @@@ ^pkg:maven/org\.yaml/snakeyaml@.*$ + CVE-2023-2251 -CVE-2022-38752 -CVE-2022-38751 -CVE-2022-38750 -CVE-2022-41854 +CVE-2017-18640 CVE-2022-25857 CVE-2022-38749 +CVE-2022-38750 +CVE-2022-38751 +CVE-2022-38752 +CVE-2022-41854 CVE-2022-1471 CVE-2022-3064 CVE-2021-4235 diff --cc CHANGES.txt index d80d0be71e,371ba6c526..b132527d5c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,9 -1,5 +1,13 @@@ -3.0.29 ++3.11.16 ++Merged from 3.0: + * Suppress CVE-2023-2251 (CASSANDRA-18497) ++ +3.11.15 + * Fix the capital P usage in the CQL parser (CASSANDRA-17919) + * Fix sstable_count metric missing from tablestats json/yaml output (CASSANDRA-18448) + * Suppress CVE-2022-45688 (CASSANDRA-18389) + * Fix Splitter sometimes creating more splits than requested (CASSANDRA-18013) +Merged from 3.0: * Do not remove SSTables when cause of FSReadError is OutOfMemoryError while using best_effort disk failure policy (CASSANDRA-18336) * Do not remove truncated_at entry in system.local while dropping an index (CASSANDRA-18105) * Save host id to system.local and flush immediately after startup (CASSANDRA-18153) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: Suppress CVE-2023-2251
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-3.0 by this push: new 4a62757624 Suppress CVE-2023-2251 4a62757624 is described below commit 4a62757624d120a0e493d9d39bcd9bf8282b4f58 Author: Brandon Williams AuthorDate: Thu May 4 09:12:18 2023 -0500 Suppress CVE-2023-2251 Patch by brandonwilliams; reviewed by smiklosovic for CASSANDRA-18497 --- .build/dependency-check-suppressions.xml | 1 + CHANGES.txt | 1 + 2 files changed, 2 insertions(+) diff --git a/.build/dependency-check-suppressions.xml b/.build/dependency-check-suppressions.xml index 4438f2259e..5a87f57c3f 100644 --- a/.build/dependency-check-suppressions.xml +++ b/.build/dependency-check-suppressions.xml @@ -23,6 +23,7 @@ ^pkg:maven/org\.yaml/snakeyaml@.*$ +CVE-2023-2251 CVE-2022-38752 CVE-2022-38751 CVE-2022-38750 diff --git a/CHANGES.txt b/CHANGES.txt index 0834f18117..371ba6c526 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.29 + * Suppress CVE-2023-2251 (CASSANDRA-18497) * Do not remove SSTables when cause of FSReadError is OutOfMemoryError while using best_effort disk failure policy (CASSANDRA-18336) * Do not remove truncated_at entry in system.local while dropping an index (CASSANDRA-18105) * Save host id to system.local and flush immediately after startup (CASSANDRA-18153) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18499) Test Failure: upgrade_through_versions_test.py::TestUpgrade::test_rolling_upgrade_with_internode_ssl
[
https://issues.apache.org/jira/browse/CASSANDRA-18499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-18499:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
> Test Failure:
> upgrade_through_versions_test.py::TestUpgrade::test_rolling_upgrade_with_internode_ssl
>
>
> Key: CASSANDRA-18499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18499
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Andres de la Peña
>Priority: Normal
>
> The upgrade Python dtest
> {{upgrade_through_versions_test.py::TestUpgrade::test_rolling_upgrade_with_internode_ssl}}
> seems to fail on CircleCI at least for trunk:
> *
> [https://app.circleci.com/pipelines/github/adelapena/cassandra/2882/workflows/b9abc2b2-2e79-47b2-b7ce-c549e75293bc/jobs/42698/tests]
> {code:java}
> self =
> object at 0x7f89fc0f17f0>
> @pytest.mark.timeout(3000)
> def test_rolling_upgrade_with_internode_ssl(self):
> """
> Rolling upgrade test using internode ssl.
> """
> > self.upgrade_scenario(rolling=True, internode_ssl=True)
> upgrade_tests/upgrade_through_versions_test.py:371:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> object at 0x7f89fc0f17f0>
> populate = True, create_schema = True, rolling = True, after_upgrade_call = ()
> internode_ssl = True
> def upgrade_scenario(self, populate=True, create_schema=True,
> rolling=False, after_upgrade_call=(), internode_ssl=False):
> # Record the rows we write as we go:
> if populate:
> self.prepare()
> self.row_values = set()
> cluster = self.cluster
> if cluster.version() >= '3.0':
>
> cluster.set_configuration_options({'enable_user_defined_functions': 'true',
>
> 'enable_scripted_user_defined_functions': 'true'})
> elif cluster.version() >= '2.2':
>
> cluster.set_configuration_options({'enable_user_defined_functions': 'true'})
>
> if internode_ssl:
> logger.debug("***using internode ssl***")
> generate_ssl_stores(self.fixture_dtest_setup.test_path)
>
> self.cluster.enable_internode_ssl(self.fixture_dtest_setup.test_path)
>
> if populate:
> # Start with 3 node cluster
> logger.debug('Creating cluster (%s)' %
> self.test_version_metas[0].version)
> cluster.populate(3)
> [node.start(use_jna=True, wait_for_binary_proto=True) for node in
> cluster.nodelist()]
> else:
> logger.debug("Skipping cluster creation (should already be
> built)")
>
> # add nodes to self for convenience
> for i, node in enumerate(cluster.nodelist(), 1):
> node_name = 'node' + str(i)
> setattr(self, node_name, node)
>
> if create_schema:
> if rolling:
> self._create_schema_for_rolling()
> else:
> self._create_schema()
> else:
> logger.debug("Skipping schema creation (should already be built)")
>
> self._log_current_ver(self.test_version_metas[0])
>
> if rolling:
> # start up processes to write and verify data
> write_proc, verify_proc, verification_queue =
> self._start_continuous_write_and_verify(wait_for_rowcount=5000)
>
> # upgrade through versions
> for version_meta in self.test_version_metas[1:]:
> if version_meta.family > '3.11' and internode_ssl:
> seeds =[]
> for seed in cluster.seeds:
> > seeds.append(seed.ip_addr + ':7001')
> E AttributeError: 'str' object has no attribute
> 'ip_addr'
> upgrade_tests/upgrade_through_versions_test.py:422: AttributeError
> {code}
> I haven't seen this failure on Jenkins yet.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18497) snakeyaml vulnerability: CVE-2023-2251
[ https://issues.apache.org/jira/browse/CASSANDRA-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719384#comment-17719384 ] Stefan Miklosovic commented on CASSANDRA-18497: --- +1 > snakeyaml vulnerability: CVE-2023-2251 > -- > > Key: CASSANDRA-18497 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18497 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > > This is failing the OWASP scan. > https://nvd.nist.gov/vuln/detail/CVE-2023-2251 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18499) Test Failure: upgrade_through_versions_test.py::TestUpgrade::test_rolling_upgrade_with_internode_ssl
Andres de la Peña created CASSANDRA-18499:
-
Summary: Test Failure:
upgrade_through_versions_test.py::TestUpgrade::test_rolling_upgrade_with_internode_ssl
Key: CASSANDRA-18499
URL: https://issues.apache.org/jira/browse/CASSANDRA-18499
Project: Cassandra
Issue Type: Bug
Components: Test/dtest/python
Reporter: Andres de la Peña
The upgrade Python dtest
{{upgrade_through_versions_test.py::TestUpgrade::test_rolling_upgrade_with_internode_ssl}}
seems to fail on CircleCI at least for trunk:
*
[https://app.circleci.com/pipelines/github/adelapena/cassandra/2882/workflows/b9abc2b2-2e79-47b2-b7ce-c549e75293bc/jobs/42698/tests]
{code:java}
self =
@pytest.mark.timeout(3000)
def test_rolling_upgrade_with_internode_ssl(self):
"""
Rolling upgrade test using internode ssl.
"""
> self.upgrade_scenario(rolling=True, internode_ssl=True)
upgrade_tests/upgrade_through_versions_test.py:371:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
populate = True, create_schema = True, rolling = True, after_upgrade_call = ()
internode_ssl = True
def upgrade_scenario(self, populate=True, create_schema=True,
rolling=False, after_upgrade_call=(), internode_ssl=False):
# Record the rows we write as we go:
if populate:
self.prepare()
self.row_values = set()
cluster = self.cluster
if cluster.version() >= '3.0':
cluster.set_configuration_options({'enable_user_defined_functions':
'true',
'enable_scripted_user_defined_functions': 'true'})
elif cluster.version() >= '2.2':
cluster.set_configuration_options({'enable_user_defined_functions':
'true'})
if internode_ssl:
logger.debug("***using internode ssl***")
generate_ssl_stores(self.fixture_dtest_setup.test_path)
self.cluster.enable_internode_ssl(self.fixture_dtest_setup.test_path)
if populate:
# Start with 3 node cluster
logger.debug('Creating cluster (%s)' %
self.test_version_metas[0].version)
cluster.populate(3)
[node.start(use_jna=True, wait_for_binary_proto=True) for node in
cluster.nodelist()]
else:
logger.debug("Skipping cluster creation (should already be built)")
# add nodes to self for convenience
for i, node in enumerate(cluster.nodelist(), 1):
node_name = 'node' + str(i)
setattr(self, node_name, node)
if create_schema:
if rolling:
self._create_schema_for_rolling()
else:
self._create_schema()
else:
logger.debug("Skipping schema creation (should already be built)")
self._log_current_ver(self.test_version_metas[0])
if rolling:
# start up processes to write and verify data
write_proc, verify_proc, verification_queue =
self._start_continuous_write_and_verify(wait_for_rowcount=5000)
# upgrade through versions
for version_meta in self.test_version_metas[1:]:
if version_meta.family > '3.11' and internode_ssl:
seeds =[]
for seed in cluster.seeds:
> seeds.append(seed.ip_addr + ':7001')
E AttributeError: 'str' object has no attribute 'ip_addr'
upgrade_tests/upgrade_through_versions_test.py:422: AttributeError
{code}
I haven't seen this failure on Jenkins yet.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18497) snakeyaml vulnerability: CVE-2023-2251
[ https://issues.apache.org/jira/browse/CASSANDRA-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18497: - Test and Documentation Plan: run CI Status: Patch Available (was: Open) > snakeyaml vulnerability: CVE-2023-2251 > -- > > Key: CASSANDRA-18497 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18497 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > > This is failing the OWASP scan. > https://nvd.nist.gov/vuln/detail/CVE-2023-2251 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18495) Warnings when using the perl driver to connect to Cassandra
[
https://issues.apache.org/jira/browse/CASSANDRA-18495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719376#comment-17719376
]
Brandon Williams commented on CASSANDRA-18495:
--
If you use a fully qualified statement such as "INSERT into ks.table ..." then
there is no need for USE or to choose a keyspace beforehand, however the
warning is only emitted for prepared statements. Unprepared statements can
still USE for example in interactive applications, like cqlsh.
> Warnings when using the perl driver to connect to Cassandra
> ---
>
> Key: CASSANDRA-18495
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18495
> Project: Cassandra
> Issue Type: Bug
>Reporter: Aswin Karthik
>Priority: Normal
>
> When I use the [perl driver|https://github.com/TvdW/perl-DBD-Cassandra] to
> connect to Cassandra 4.0.3 and onwards, I get the following error just for
> initialization of a connection.
> The error I get is
> {noformat}
> `USE ` with prepared statements is considered to be an anti-pattern
> due to ambiguity in non-qualified table names. Please consider removing
> instances of `Session#setKeyspace()`, `Session#execute("USE
> ")` and `cluster.newSession()` from your code, and always
> use fully qualified table names (e.g. .). Keyspace used:
> null, statement keyspace: null, statement id: at
> Cassandra/Client/Connection.pm line 957.
> {noformat}
>
> This is just from initialization of the perl driver connection while choosing
> the keyspace.
> [https://github.com/TvdW/perl-DBD-Cassandra/blob/master/Cassandra-Client/lib/Cassandra/Client/Connection.pm#L562]
> before running any queries. It is emitted even if I use qualified prepared
> statement.
> The same warning does not pop up in Datastax java driver initialization.
> On debugging, I found that this warning does not emit for all unqualified
> prepared statements. It only emits for unqualified prepared "USE ks"
> statement. But the "USE ks" can never be qualified. So the warning is a bit
> vague on what is the recommended approach.
> And from the perspective of the driver, it is setting the keyspace of the
> connection for the first time. The same warning does not happen on the
> datastax java driver and that is because it uses QUERY to set the keyspace on
> connection. (I tried to follow the same approach on the perl driver -
> https://github.com/TvdW/perl-DBD-Cassandra/pull/35 )
> The warnings are not very clear on what is deprecated and what is not. Does
> it deprecate only the use of prepared statement of "USE ks"? or does it
> deprecate "USE ks" completely? And it is not being emitted for other
> unqualified prepared statements but only for a USE statement which cannot be
> qualified at all.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18498) Add option to sstablepartitions for aggregating stats
Andres de la Peña created CASSANDRA-18498:
-
Summary: Add option to sstablepartitions for aggregating stats
Key: CASSANDRA-18498
URL: https://issues.apache.org/jira/browse/CASSANDRA-18498
Project: Cassandra
Issue Type: Improvement
Components: Tool/sstable
Reporter: Andres de la Peña
Assignee: Andres de la Peña
The {{sstablepartitions}} offline tool added by CASSANDRA-8720 processes
sstables one by one. If a partition is spread across multiple sstables, the
tool will separately report the metrics of the partition fragment on each
sstable. For example, if we are looking for partitions larger than 100MiB, and
a partition has a 50MiB fragment in one table and 50MiB in another table, the
partition won't be found.
It would be useful to add a kind of {{--aggregate}} option to
{{sstablepartitions}} so it aggregates the results of all the analyzed sstables
that belong to the same table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17918) DESCRIBE output does not quote column names using reserved keywords
[
https://issues.apache.org/jira/browse/CASSANDRA-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719367#comment-17719367
]
Stefan Miklosovic edited comment on CASSANDRA-17918 at 5/4/23 2:49 PM:
---
The problem with tests is that the patch introduces this result of aggregate
description, for example it would describe an aggregate like this:
{code:java}
CREATE AGGREGATE cqlshtests_dligxmowmx.aggmax(int)
SFUNC cqlshtests_dligxmowmx."fmax"
STYPE int; {code}
This does not work because the the value of SFUNC is in form "keyspace.name"
and it does not accept this. There can not be ".". It just expects name of the
function.
[https://github.com/apache/cassandra/blob/trunk/src/antlr/Parser.g#L671]
[https://github.com/apache/cassandra/blob/trunk/src/antlr/Parser.g#L1567]
So the original author introduced something to CQL but parser was not modified.
I do not know how to fix this. I think we should just leave it out as it was
and we should just quote the function name. If we are creating an aggregate on
some keyspace, I do not think that SFUNC is going to ever be something else
from a function in that keyspace. So the further distinction what keyspace that
SFUNC is from does not make a lot of sense.
my PR for trunk is here which builds on top of the original work:
https://github.com/apache/cassandra/pull/2308
was (Author: smiklosovic):
The problem with tests is that the patch introduces this result of aggregate
description, for example it would describe an aggregate like this:
{code:java}
CREATE AGGREGATE cqlshtests_dligxmowmx.aggmax(int)
SFUNC cqlshtests_dligxmowmx."fmax"
STYPE int; {code}
This does not work because the the value of SFUNC is in form "keyspace.name"
and it does not accept this. There can not be ".". It just expects name of the
function.
[https://github.com/apache/cassandra/blob/trunk/src/antlr/Parser.g#L671]
[https://github.com/apache/cassandra/blob/trunk/src/antlr/Parser.g#L1567]
So the original author introduced something to CQL but parser was not modified.
I do not know how to fix this. I think we should just leave it out as it was
and we should just quote the function name. If we are creating an aggregate on
some keyspace, I do not think that SFUNC is going to ever be something else
from a function in that keyspace. So the further distinction what keyspace that
SFUNC is from does not make a lot of sense.
> DESCRIBE output does not quote column names using reserved keywords
> ---
>
> Key: CASSANDRA-17918
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17918
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
>
> The DESCRIBE output of the column names that using reserved keywords are not
> quoted for UDTs. The following test reproduces. Reading the code, it looks
> like that the such columns names are not quoted in materialized view, UDF and
> user defined aggregation.
> The impact of the bug is that schema described cannot be imported due to the
> usage of reserved keywords as column names.
>
> {code:java}
> @Test
> public void testUsingReservedInCreateType() throws Throwable
> {
> String type = createType(KEYSPACE_PER_TEST, "CREATE TYPE %s
> (\"token\" text, \"desc\" text);");
> assertRowsNet(executeDescribeNet(KEYSPACE_PER_TEST, "DESCRIBE TYPE " + type),
> row(KEYSPACE_PER_TEST, "type", type, "CREATE TYPE " +
> KEYSPACE_PER_TEST + "." + type + " (\n" +
> " \"token\" text,\n" +
> " \"desc\" text\n" +
> ");"));
> } {code}
> +Additional information for newcomers:+
> * Unit tests for DESCRIBE statements are in {{DescribeStatementTest}}
> * The statement implementation is in {{DescribeStatement and fetch the
> create statement from the different schema element using
> SchemaElement.toCqlString}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17918) DESCRIBE output does not quote column names using reserved keywords
[
https://issues.apache.org/jira/browse/CASSANDRA-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719367#comment-17719367
]
Stefan Miklosovic commented on CASSANDRA-17918:
---
The problem with tests is that the patch introduces this result of aggregate
description, for example it would describe an aggregate like this:
{code:java}
CREATE AGGREGATE cqlshtests_dligxmowmx.aggmax(int)
SFUNC cqlshtests_dligxmowmx."fmax"
STYPE int; {code}
This does not work because the the value of SFUNC is in form "keyspace.name"
and it does not accept this. There can not be ".". It just expects name of the
function.
[https://github.com/apache/cassandra/blob/trunk/src/antlr/Parser.g#L671]
[https://github.com/apache/cassandra/blob/trunk/src/antlr/Parser.g#L1567]
So the original author introduced something to CQL but parser was not modified.
I do not know how to fix this. I think we should just leave it out as it was
and we should just quote the function name. If we are creating an aggregate on
some keyspace, I do not think that SFUNC is going to ever be something else
from a function in that keyspace. So the further distinction what keyspace that
SFUNC is from does not make a lot of sense.
> DESCRIBE output does not quote column names using reserved keywords
> ---
>
> Key: CASSANDRA-17918
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17918
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
>
> The DESCRIBE output of the column names that using reserved keywords are not
> quoted for UDTs. The following test reproduces. Reading the code, it looks
> like that the such columns names are not quoted in materialized view, UDF and
> user defined aggregation.
> The impact of the bug is that schema described cannot be imported due to the
> usage of reserved keywords as column names.
>
> {code:java}
> @Test
> public void testUsingReservedInCreateType() throws Throwable
> {
> String type = createType(KEYSPACE_PER_TEST, "CREATE TYPE %s
> (\"token\" text, \"desc\" text);");
> assertRowsNet(executeDescribeNet(KEYSPACE_PER_TEST, "DESCRIBE TYPE " + type),
> row(KEYSPACE_PER_TEST, "type", type, "CREATE TYPE " +
> KEYSPACE_PER_TEST + "." + type + " (\n" +
> " \"token\" text,\n" +
> " \"desc\" text\n" +
> ");"));
> } {code}
> +Additional information for newcomers:+
> * Unit tests for DESCRIBE statements are in {{DescribeStatementTest}}
> * The statement implementation is in {{DescribeStatement and fetch the
> create statement from the different schema element using
> SchemaElement.toCqlString}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18472) Docker images can no longer be built due to virtualenv from pip
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719360#comment-17719360
]
Brandon Williams edited comment on CASSANDRA-18472 at 5/4/23 2:33 PM:
--
bq. You only changed it in one place, not on the whole file.
Argh, you caught me! :) Let me fix that also
bq. 3.0/3.11 seems to have python-ish failures that don't align to what I see
for 3.11 and 3.0 in jenkins
Those were temporary failures from some other patch that should now be
resolved, so let's just try again:
||Branch||CI||
|[3.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18472-3.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1000/workflows/6a1490e5-0328-4d9a-b051-99be2d3490c5]|
|[3.11|https://github.com/driftx/cassandra/tree/CASSANDRA-18472-3.11]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/999/workflows/756643b4-2cae-4d81-8be9-0af81cc2f9dd]|
was (Author: brandon.williams):
bq. You only changed it in one place, not on the whole file.
Argh, you caught me! :) Let me fix that also
bq. 3.0/3.11 seems to have python-ish failures that don't align to what I see
for 3.11 and 3.0 in jenkins
Those were temporary failures from some other patch that should now be
resolved, so let's just try again:
||Branch||CI||
|[3.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18472-3.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/982/workflows/9240e8b6-bc0e-4fb6-8081-8ec1eef3a6a7]|
|[3.11|https://github.com/driftx/cassandra/tree/CASSANDRA-18472-3.11]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/999/workflows/756643b4-2cae-4d81-8be9-0af81cc2f9dd]|
> Docker images can no longer be built due to virtualenv from pip
> ---
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18472) Docker images can no longer be built due to virtualenv from pip
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719360#comment-17719360
]
Brandon Williams commented on CASSANDRA-18472:
--
bq. You only changed it in one place, not on the whole file.
Argh, you caught me! :) Let me fix that also
bq. 3.0/3.11 seems to have python-ish failures that don't align to what I see
for 3.11 and 3.0 in jenkins
Those were temporary failures from some other patch that should now be
resolved, so let's just try again:
||Branch||CI||
|[3.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18472-3.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/982/workflows/9240e8b6-bc0e-4fb6-8081-8ec1eef3a6a7]|
|[3.11|https://github.com/driftx/cassandra/tree/CASSANDRA-18472-3.11]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/999/workflows/756643b4-2cae-4d81-8be9-0af81cc2f9dd]|
> Docker images can no longer be built due to virtualenv from pip
> ---
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18497) snakeyaml vulnerability: CVE-2023-2251
[ https://issues.apache.org/jira/browse/CASSANDRA-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719351#comment-17719351 ] Brandon Williams commented on CASSANDRA-18497: -- ||Branch||CI|| |[3.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18497-3.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/994/workflows/76de842e-e8d7-4553-9de1-e514e051730d]| |[3.11|https://github.com/driftx/cassandra/tree/CASSANDRA-18497-3.11]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/996/workflows/f34aef4c-a924-4d27-becb-c1bccd21d97e]| |[4.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18497-4.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/997/workflows/b1c380ae-32e6-4f19-b417-1f715cb9578b], [j11|https://app.circleci.com/pipelines/github/driftx/cassandra/997/workflows/37a33da1-6b78-4de4-b2b1-7ae2fdae2048]| |[4.1|https://github.com/driftx/cassandra/tree/CASSANDRA-18497-4.1]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/995/workflows/49dfa2df-43d3-45b6-9004-d8e7f627dbe0], [j11|https://app.circleci.com/pipelines/github/driftx/cassandra/995/workflows/1c516cd6-7002-4d61-8436-755fa96a9947]| |[trunk|https://github.com/driftx/cassandra/tree/CASSANDRA-18497-trunk]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/998/workflows/35f5da30-4789-4715-81e1-0adb3e4b90df], [j11|https://app.circleci.com/pipelines/github/driftx/cassandra/998/workflows/ce40ec05-f1fe-4ece-b88e-49b2c40fce00]| > snakeyaml vulnerability: CVE-2023-2251 > -- > > Key: CASSANDRA-18497 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18497 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > > This is failing the OWASP scan. > https://nvd.nist.gov/vuln/detail/CVE-2023-2251 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18467) Update generate-idea-files for J17
[ https://issues.apache.org/jira/browse/CASSANDRA-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719345#comment-17719345 ] Ekaterina Dimitrova edited comment on CASSANDRA-18467 at 5/4/23 2:11 PM: - Oh, this broke now tests in IntelliJ with JDK17 I can take a look at it tomorrow, multi tasking doesn't work on my end today was (Author: e.dimitrova): Oh, this broke now tests in IntelliJ with JDK17 > Update generate-idea-files for J17 > -- > > Key: CASSANDRA-18467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18467 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Jakub Zytka >Priority: Low > Fix For: 5.x > > > There was a discussion in CASSANDRA-18258 how to update generate-idea-files. > The final agreement was to create one target to cover both Java 11 and Java > 17. > It will be good to figure out CASSANDRA-18263 and reshuffle arguments and > tasks based on what we decide to use as gc in testing for both Java 11 and > Java 17. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18497) snakeyaml vulnerability: CVE-2023-2251
[ https://issues.apache.org/jira/browse/CASSANDRA-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719347#comment-17719347 ] Brandon Williams commented on CASSANDRA-18497: -- I think the hit for snakeyaml is a false positive, this is for [this repo|https://github.com/eemeli/yaml] which is a javascript yaml parser. We can simply suppress. > snakeyaml vulnerability: CVE-2023-2251 > -- > > Key: CASSANDRA-18497 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18497 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > > This is failing the OWASP scan. > https://nvd.nist.gov/vuln/detail/CVE-2023-2251 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18467) Update generate-idea-files for J17
[ https://issues.apache.org/jira/browse/CASSANDRA-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719345#comment-17719345 ] Ekaterina Dimitrova commented on CASSANDRA-18467: - Oh, this broke now tests in IntelliJ with JDK17 > Update generate-idea-files for J17 > -- > > Key: CASSANDRA-18467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18467 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Jakub Zytka >Priority: Low > Fix For: 5.x > > > There was a discussion in CASSANDRA-18258 how to update generate-idea-files. > The final agreement was to create one target to cover both Java 11 and Java > 17. > It will be good to figure out CASSANDRA-18263 and reshuffle arguments and > tasks based on what we decide to use as gc in testing for both Java 11 and > Java 17. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18497) snakeyaml vulnerability: CVE-2023-2251
[ https://issues.apache.org/jira/browse/CASSANDRA-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18497: - Bug Category: Parent values: Security(12985) Complexity: Normal Component/s: Local/Config Discovered By: User Report Fix Version/s: 3.0.x 3.11.x 4.0.x 4.1.x 5.x Severity: Normal Assignee: Brandon Williams Status: Open (was: Triage Needed) > snakeyaml vulnerability: CVE-2023-2251 > -- > > Key: CASSANDRA-18497 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18497 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > > This is failing the OWASP scan. > https://nvd.nist.gov/vuln/detail/CVE-2023-2251 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18497) snakeyaml vulnerability: CVE-2023-2251
Brandon Williams created CASSANDRA-18497: Summary: snakeyaml vulnerability: CVE-2023-2251 Key: CASSANDRA-18497 URL: https://issues.apache.org/jira/browse/CASSANDRA-18497 Project: Cassandra Issue Type: Bug Reporter: Brandon Williams This is failing the OWASP scan. https://nvd.nist.gov/vuln/detail/CVE-2023-2251 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12937) Default setting (yaml) for SSTable compression
[
https://issues.apache.org/jira/browse/CASSANDRA-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719320#comment-17719320
]
Michael Semb Wever commented on CASSANDRA-12937:
bq. I am also little bit puzzled by the git workflow here, all I see is that
you "merged" trunk into that branch. When I checkout that I do not see the
history so I am not sure how we want to squash and merge this.
Yeah, please reserve merges only for forward-merging from release branches.
Feature/dev branches should be rebased.
> Default setting (yaml) for SSTable compression
> --
>
> Key: CASSANDRA-12937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12937
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Michael Semb Wever
>Assignee: Claude Warren
>Priority: Low
> Labels: AdventCalendar2021, lhf
> Fix For: 5.x
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> In many situations the choice of compression for sstables is more relevant to
> the disks attached than to the schema and data.
> This issue is to add to cassandra.yaml a default value for sstable
> compression that new tables will inherit (instead of the defaults found in
> {{CompressionParams.DEFAULT}}.
> Examples where this can be relevant are filesystems that do on-the-fly
> compression (btrfs, zfs) or specific disk configurations or even specific C*
> versions (see CASSANDRA-10995 ).
> +Additional information for newcomers+
> Some new fields need to be added to {{cassandra.yaml}} to allow specifying
> the field required for defining the default compression parameters. In
> {{DatabaseDescriptor}} a new {{CompressionParams}} field should be added for
> the default compression. This field should be initialized in
> {{DatabaseDescriptor.applySimpleConfig()}}. At the different places where
> {{CompressionParams.DEFAULT}} was used the code should call
> {{DatabaseDescriptor#getDefaultCompressionParams}} that should return some
> copy of configured {{CompressionParams}}.
> Some unit test using {{OverrideConfigurationLoader}} should be used to test
> that the table schema use the new default when a new table is created (see
> CreateTest for some example).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18467) Update generate-idea-files for J17
[
https://issues.apache.org/jira/browse/CASSANDRA-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719316#comment-17719316
]
Ekaterina Dimitrova commented on CASSANDRA-18467:
-
CI runs show that we need actually to assign empty string to the jvm args for
JDK8 as it was before. It just becomes visible only running tests with ant from
command line. They pass in IntelliJ. Ninja fixed here -
[https://github.com/ekaterinadimitrova2/cassandra/commit/67af4975a73de978aa883e9cc8e03226a124b099]
Restarted CI with 8 and 11.
{color:#7a869a}bq. I believe it was a wise way to keep things simple{color}
Totally agree, I believe there are also other places where we need to take
similar approach but this can be done incrementally.
> Update generate-idea-files for J17
> --
>
> Key: CASSANDRA-18467
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18467
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Ekaterina Dimitrova
>Assignee: Jakub Zytka
>Priority: Low
> Fix For: 5.x
>
>
> There was a discussion in CASSANDRA-18258 how to update generate-idea-files.
> The final agreement was to create one target to cover both Java 11 and Java
> 17.
> It will be good to figure out CASSANDRA-18263 and reshuffle arguments and
> tasks based on what we decide to use as gc in testing for both Java 11 and
> Java 17.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Add sstablepartitions offline tool to find large partitions in sstables
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new e72ec4e828 Add sstablepartitions offline tool to find large partitions in sstables e72ec4e828 is described below commit e72ec4e82835cef80cf9e9d8c58b25801d4af8c8 Author: Andrés de la Peña AuthorDate: Fri Apr 21 18:31:32 2023 +0100 Add sstablepartitions offline tool to find large partitions in sstables patch by Andrés de la Peña and Robert Stupp; reviewed by Brandon Williams and Stefan Miklosovic for CASSANDRA-8720 Co-authored-by: Andrés de la Peña Co-authored-by: Robert Stupp --- CHANGES.txt| 1 + NEWS.txt | 1 + debian/cassandra.install | 1 + .../pages/managing/tools/sstable/index.adoc| 1 + .../managing/tools/sstable/sstablepartitions.adoc | 171 redhat/cassandra.spec | 1 + .../apache/cassandra/io/sstable/Descriptor.java| 43 +- src/java/org/apache/cassandra/schema/TableId.java | 25 + .../apache/cassandra/tools/SSTablePartitions.java | 889 + .../cassandra/tools/SSTablePartitionsTest.java | 654 +++ tools/bin/sstablepartitions| 49 ++ 11 files changed, 1823 insertions(+), 13 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 862755286a..970058c239 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 5.0 + * Add sstablepartitions offline tool to find large partitions in sstables (CASSANDRA-8720) * Replace usages of json-simple dependency by Jackson (CASSANDRA-16855) * When decommissioning should set Severity to limit traffic (CASSANDRA-18430) * For Java11 and Java17 remove -XX:-UseBiasedLocking as it is the default already (CASSANDRA-17869) diff --git a/NEWS.txt b/NEWS.txt index b18a5e3a48..4e0792aee6 100644 --- a/NEWS.txt +++ b/NEWS.txt @@ -140,6 +140,7 @@ New features - Added new SELECT_MASKED permission. It allows to run SELECT queries selecting the clear values of masked columns. Superusers have it by default, whereas regular users don't have it by default. - Added support for using UDFs as masking functions attached to table columns on the schema. +- Added `sstablepartitions` offline tool to find large partitions in sstables. Upgrading - diff --git a/debian/cassandra.install b/debian/cassandra.install index f54d1ad835..dced5a29e2 100644 --- a/debian/cassandra.install +++ b/debian/cassandra.install @@ -25,6 +25,7 @@ tools/bin/fqltool usr/bin tools/bin/auditlogviewer usr/bin tools/bin/jmxtool usr/bin tools/bin/hash_password usr/bin +tools/bin/sstablepartitions usr/bin lib/*.jar usr/share/cassandra/lib lib/*.zip usr/share/cassandra/lib lib/sigar-bin/* usr/share/cassandra/lib/sigar-bin diff --git a/doc/modules/cassandra/pages/managing/tools/sstable/index.adoc b/doc/modules/cassandra/pages/managing/tools/sstable/index.adoc index cb787ece21..cc7637c8dd 100644 --- a/doc/modules/cassandra/pages/managing/tools/sstable/index.adoc +++ b/doc/modules/cassandra/pages/managing/tools/sstable/index.adoc @@ -12,6 +12,7 @@ stopped. * xref:tools/sstable/sstableloader.adoc[sstableloader] * xref:tools/sstable/sstablemetadata.adoc[sstablemetadata] * xref:tools/sstable/sstableofflinerelevel.adoc[sstableofflinerelevel] +* xref:tools/sstable/sstablepartitions.adoc[sstablepartitions] * xref:tools/sstable/sstablerepairedset.adoc[sstablerepairdset] * xref:tools/sstable/sstablescrub.adoc[sstablescrub] * xref:tools/sstable/sstablesplit.adoc[sstablesplit] diff --git a/doc/modules/cassandra/pages/managing/tools/sstable/sstablepartitions.adoc b/doc/modules/cassandra/pages/managing/tools/sstable/sstablepartitions.adoc new file mode 100644 index 00..6684946b47 --- /dev/null +++ b/doc/modules/cassandra/pages/managing/tools/sstable/sstablepartitions.adoc @@ -0,0 +1,171 @@ += sstablepartitions + +Identifies large partitions of SSTables and outputs the partition size in bytes, row count, cell count, and tombstone count. + +You can supply any number of sstables file paths, or directories containing sstables. Each sstable will be analyzed separately. + +If a metrics threshold such as `--min-size`, `--min-rows`, `--min-cells` or `--min-tombstones` is provided, +then the partition keys exceeding of the threshold will be printed in the output. +It also prints a summary of metrics for the table. The percentiles in the metrics are estimates, +while the min/max/count metrics are accurate. + +The default output of this tool is meant to be read by human eyes. +Future versions might include small formatting changes or present new data that can fool scripts reading it. +Scripts or other automatic tools should use the `--csv` flag to produce machine-rea
[jira] [Updated] (CASSANDRA-18488) WEBSITE - Replace homepage, Events page banners with C* Summit
[ https://issues.apache.org/jira/browse/CASSANDRA-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Ramirez updated CASSANDRA-18488: -- Summary: WEBSITE - Replace homepage, Events page banners with C* Summit (was: WEBSITE - Replace Forward banner with Summit banner on pages) > WEBSITE - Replace homepage, Events page banners with C* Summit > -- > > Key: CASSANDRA-18488 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18488 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Stefano Lottini >Assignee: Stefano Lottini >Priority: Normal > Labels: pull-request-available > Fix For: NA > > Attachments: banner_on_events_page.png, banner_on_index_page.png > > > Cassandra Forward is in the past, so this addresses the fact by simply > replacing the banner and the "register now" CTA with an equivalent thing for > the December C* Summit. > The (very simple) PR is available for inspection here: > [https://github.com/apache/cassandra-website/pull/219] > See images: > !banner_on_index_page.png|width=300! > !banner_on_events_page.png|width=300! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18488) WEBSITE - Replace Forward banner with Summit banner on pages
[ https://issues.apache.org/jira/browse/CASSANDRA-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Ramirez updated CASSANDRA-18488: -- Status: Changes Suggested (was: Review In Progress) The PR looks good to me but I do agree with Mick that we shouldn't delete the old C* Fwd banner for posterity. Cheers! > WEBSITE - Replace Forward banner with Summit banner on pages > > > Key: CASSANDRA-18488 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18488 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Stefano Lottini >Assignee: Stefano Lottini >Priority: Normal > Labels: pull-request-available > Fix For: NA > > Attachments: banner_on_events_page.png, banner_on_index_page.png > > > Cassandra Forward is in the past, so this addresses the fact by simply > replacing the banner and the "register now" CTA with an equivalent thing for > the December C* Summit. > The (very simple) PR is available for inspection here: > [https://github.com/apache/cassandra-website/pull/219] > See images: > !banner_on_index_page.png|width=300! > !banner_on_events_page.png|width=300! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18488) WEBSITE - Replace Forward banner with Summit banner on pages
[ https://issues.apache.org/jira/browse/CASSANDRA-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Ramirez updated CASSANDRA-18488: -- Test and Documentation Plan: Stage locally and verify that the updated pages render as expected Status: Patch Available (was: In Progress) > WEBSITE - Replace Forward banner with Summit banner on pages > > > Key: CASSANDRA-18488 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18488 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Stefano Lottini >Assignee: Stefano Lottini >Priority: Normal > Labels: pull-request-available > Fix For: NA > > Attachments: banner_on_events_page.png, banner_on_index_page.png > > > Cassandra Forward is in the past, so this addresses the fact by simply > replacing the banner and the "register now" CTA with an equivalent thing for > the December C* Summit. > The (very simple) PR is available for inspection here: > [https://github.com/apache/cassandra-website/pull/219] > See images: > !banner_on_index_page.png|width=300! > !banner_on_events_page.png|width=300! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18488) WEBSITE - Replace Forward banner with Summit banner on pages
[ https://issues.apache.org/jira/browse/CASSANDRA-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Ramirez updated CASSANDRA-18488: -- Status: Review In Progress (was: Patch Available) > WEBSITE - Replace Forward banner with Summit banner on pages > > > Key: CASSANDRA-18488 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18488 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Stefano Lottini >Assignee: Stefano Lottini >Priority: Normal > Labels: pull-request-available > Fix For: NA > > Attachments: banner_on_events_page.png, banner_on_index_page.png > > > Cassandra Forward is in the past, so this addresses the fact by simply > replacing the banner and the "register now" CTA with an equivalent thing for > the December C* Summit. > The (very simple) PR is available for inspection here: > [https://github.com/apache/cassandra-website/pull/219] > See images: > !banner_on_index_page.png|width=300! > !banner_on_events_page.png|width=300! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18488) WEBSITE - Replace Forward banner with Summit banner on pages
[ https://issues.apache.org/jira/browse/CASSANDRA-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Ramirez updated CASSANDRA-18488: -- Reviewers: Erick Ramirez, Michael Semb Wever (was: Erick Ramirez) > WEBSITE - Replace Forward banner with Summit banner on pages > > > Key: CASSANDRA-18488 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18488 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Stefano Lottini >Assignee: Stefano Lottini >Priority: Normal > Labels: pull-request-available > Fix For: NA > > Attachments: banner_on_events_page.png, banner_on_index_page.png > > > Cassandra Forward is in the past, so this addresses the fact by simply > replacing the banner and the "register now" CTA with an equivalent thing for > the December C* Summit. > The (very simple) PR is available for inspection here: > [https://github.com/apache/cassandra-website/pull/219] > See images: > !banner_on_index_page.png|width=300! > !banner_on_events_page.png|width=300! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18488) WEBSITE - Replace Forward banner with Summit banner on pages
[ https://issues.apache.org/jira/browse/CASSANDRA-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRA-18488: --- Labels: pull-request-available (was: ) > WEBSITE - Replace Forward banner with Summit banner on pages > > > Key: CASSANDRA-18488 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18488 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Stefano Lottini >Assignee: Stefano Lottini >Priority: Normal > Labels: pull-request-available > Fix For: NA > > Attachments: banner_on_events_page.png, banner_on_index_page.png > > > Cassandra Forward is in the past, so this addresses the fact by simply > replacing the banner and the "register now" CTA with an equivalent thing for > the December C* Summit. > The (very simple) PR is available for inspection here: > [https://github.com/apache/cassandra-website/pull/219] > See images: > !banner_on_index_page.png|width=300! > !banner_on_events_page.png|width=300! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18467) Update generate-idea-files for J17
[ https://issues.apache.org/jira/browse/CASSANDRA-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719295#comment-17719295 ] Michael Semb Wever commented on CASSANDRA-18467: bq. With that said I tested that the currently proposed patch not only looks good to me but also works as expected. With that said by [~e.dimitrova], I'm +1. Much appreciated this approach was taken, I believe it was a wise way to keep things simple > Update generate-idea-files for J17 > -- > > Key: CASSANDRA-18467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18467 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Jakub Zytka >Priority: Low > Fix For: 5.x > > > There was a discussion in CASSANDRA-18258 how to update generate-idea-files. > The final agreement was to create one target to cover both Java 11 and Java > 17. > It will be good to figure out CASSANDRA-18263 and reshuffle arguments and > tasks based on what we decide to use as gc in testing for both Java 11 and > Java 17. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18488) WEBSITE - Replace Forward banner with Summit banner on pages
[ https://issues.apache.org/jira/browse/CASSANDRA-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Ramirez updated CASSANDRA-18488: -- Description: Cassandra Forward is in the past, so this addresses the fact by simply replacing the banner and the "register now" CTA with an equivalent thing for the December C* Summit. The (very simple) PR is available for inspection here: [https://github.com/apache/cassandra-website/pull/219] See images: !banner_on_index_page.png|width=300! !banner_on_events_page.png! was: Cassandra Forward is in the past, so this addresses the fact by simply replacing the banner and the "register now" CTA with an equivalent thing for the December C* Summit. The (very simple) PR is available for inspection here: [https://github.com/apache/cassandra-website/pull/219] See images: !banner_on_index_page.png! > WEBSITE - Replace Forward banner with Summit banner on pages > > > Key: CASSANDRA-18488 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18488 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Stefano Lottini >Assignee: Stefano Lottini >Priority: Normal > Fix For: NA > > Attachments: banner_on_events_page.png, banner_on_index_page.png > > > Cassandra Forward is in the past, so this addresses the fact by simply > replacing the banner and the "register now" CTA with an equivalent thing for > the December C* Summit. > The (very simple) PR is available for inspection here: > [https://github.com/apache/cassandra-website/pull/219] > See images: > !banner_on_index_page.png|width=300! > !banner_on_events_page.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18488) WEBSITE - Replace Forward banner with Summit banner on pages
[ https://issues.apache.org/jira/browse/CASSANDRA-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Ramirez updated CASSANDRA-18488: -- Description: Cassandra Forward is in the past, so this addresses the fact by simply replacing the banner and the "register now" CTA with an equivalent thing for the December C* Summit. The (very simple) PR is available for inspection here: [https://github.com/apache/cassandra-website/pull/219] See images: !banner_on_index_page.png! was: Cassandra Forward is in the past, so this addresses the fact by simply replacing the banner and the "register now" CTA with an equivalent thing for the December C* Summit. The (very simple) PR is available for inspection here: https://github.com/apache/cassandra-website/pull/219 See images: > WEBSITE - Replace Forward banner with Summit banner on pages > > > Key: CASSANDRA-18488 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18488 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Stefano Lottini >Assignee: Stefano Lottini >Priority: Normal > Fix For: NA > > Attachments: banner_on_events_page.png, banner_on_index_page.png > > > Cassandra Forward is in the past, so this addresses the fact by simply > replacing the banner and the "register now" CTA with an equivalent thing for > the December C* Summit. > The (very simple) PR is available for inspection here: > [https://github.com/apache/cassandra-website/pull/219] > See images: > !banner_on_index_page.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18488) WEBSITE - Replace Forward banner with Summit banner on pages
[ https://issues.apache.org/jira/browse/CASSANDRA-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Ramirez updated CASSANDRA-18488: -- Description: Cassandra Forward is in the past, so this addresses the fact by simply replacing the banner and the "register now" CTA with an equivalent thing for the December C* Summit. The (very simple) PR is available for inspection here: [https://github.com/apache/cassandra-website/pull/219] See images: !banner_on_index_page.png|width=300! !banner_on_events_page.png|width=300! was: Cassandra Forward is in the past, so this addresses the fact by simply replacing the banner and the "register now" CTA with an equivalent thing for the December C* Summit. The (very simple) PR is available for inspection here: [https://github.com/apache/cassandra-website/pull/219] See images: !banner_on_index_page.png|width=300! !banner_on_events_page.png! > WEBSITE - Replace Forward banner with Summit banner on pages > > > Key: CASSANDRA-18488 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18488 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Stefano Lottini >Assignee: Stefano Lottini >Priority: Normal > Fix For: NA > > Attachments: banner_on_events_page.png, banner_on_index_page.png > > > Cassandra Forward is in the past, so this addresses the fact by simply > replacing the banner and the "register now" CTA with an equivalent thing for > the December C* Summit. > The (very simple) PR is available for inspection here: > [https://github.com/apache/cassandra-website/pull/219] > See images: > !banner_on_index_page.png|width=300! > !banner_on_events_page.png|width=300! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18488) WEBSITE - Replace Forward banner with Summit banner on pages
[ https://issues.apache.org/jira/browse/CASSANDRA-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Ramirez updated CASSANDRA-18488: -- Change Category: Semantic Complexity: Normal Component/s: Documentation/Website Fix Version/s: NA Reviewers: Erick Ramirez Status: Open (was: Triage Needed) > WEBSITE - Replace Forward banner with Summit banner on pages > > > Key: CASSANDRA-18488 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18488 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Stefano Lottini >Assignee: Stefano Lottini >Priority: Normal > Fix For: NA > > Attachments: banner_on_events_page.png, banner_on_index_page.png > > > Cassandra Forward is in the past, so this addresses the fact by simply > replacing the banner and the "register now" CTA with an equivalent thing for > the December C* Summit. > The (very simple) PR is available for inspection here: > https://github.com/apache/cassandra-website/pull/219 > See images: -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18467) Update generate-idea-files for J17
[ https://issues.apache.org/jira/browse/CASSANDRA-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719269#comment-17719269 ] Ekaterina Dimitrova edited comment on CASSANDRA-18467 at 5/4/23 11:04 AM: -- [~jakubzytka] helped me to figure out the issue. The Itellij Idea SDK had a few options for JDK 11 and if it gets the wrong one which doesn't work, this triggers the IDE to assign default JDK 17. It seems we might also want to add a step or at least instruction to ask people to remove .idea and .cassandra.iml before running *ant generate-idea-files.* It seems also reasonable to stress in docs to pay attention to File -> Project Structure -> SDK With that said I tested that the currently proposed patch not only looks good to me but also works as expected. I triggered CI: [8+11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=CASSANDRA-18467], [11+17|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18467-JDK17] was (Author: e.dimitrova): [~jakubzytka] helped me to figure out the issue. The Itellij Idea SDK had a few options for JDK 11 and if it gets the wrong one which doesn't work, this triggers the IDE to assign default JDK 17. It seems we might also want to add a step or at least instruction to ask people to remove .idea and .cassandra.iml before running *ant generate-idea-files* and pay attention to File -> Project Structure -> SDK With that said I tested that the currently proposed patch not only looks good to me but also works as expected. I triggered CI: [8+11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=CASSANDRA-18467], [11+17|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18467-JDK17] > Update generate-idea-files for J17 > -- > > Key: CASSANDRA-18467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18467 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Jakub Zytka >Priority: Low > Fix For: 5.x > > > There was a discussion in CASSANDRA-18258 how to update generate-idea-files. > The final agreement was to create one target to cover both Java 11 and Java > 17. > It will be good to figure out CASSANDRA-18263 and reshuffle arguments and > tasks based on what we decide to use as gc in testing for both Java 11 and > Java 17. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18467) Update generate-idea-files for J17
[ https://issues.apache.org/jira/browse/CASSANDRA-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719269#comment-17719269 ] Ekaterina Dimitrova edited comment on CASSANDRA-18467 at 5/4/23 11:03 AM: -- [~jakubzytka] helped me to figure out the issue. The Itellij Idea SDK had a few options for JDK 11 and if it gets the wrong one which doesn't work, this triggers the IDE to assign default JDK 17. It seems we might also want to add a step or at least instruction to ask people to remove .idea and .cassandra.iml before running *ant generate-idea-files* and pay attention to File -> Project Structure -> SDK With that said I tested that the currently proposed patch not only looks good to me but also works as expected. I triggered CI: [8+11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=CASSANDRA-18467], [11+17|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18467-JDK17] was (Author: e.dimitrova): [~jakubzytka] helped me to figure out the issue. The SDK had a few options for 11 and if it gets the wrong one which doesn't work, this triggers the IDE to assign default 17. It seems we might also want to add a step or at least instruction to ask people to remove .idea and .cassandra.iml before running *ant generate-idea-files* and pay attention to File -> Project Structure -> SDK With that said I tested that the currently proposed patch not only looks good to me but also works as expected. I triggered CI: [8+11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=CASSANDRA-18467], [11+17|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18467-JDK17] > Update generate-idea-files for J17 > -- > > Key: CASSANDRA-18467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18467 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Jakub Zytka >Priority: Low > Fix For: 5.x > > > There was a discussion in CASSANDRA-18258 how to update generate-idea-files. > The final agreement was to create one target to cover both Java 11 and Java > 17. > It will be good to figure out CASSANDRA-18263 and reshuffle arguments and > tasks based on what we decide to use as gc in testing for both Java 11 and > Java 17. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18467) Update generate-idea-files for J17
[ https://issues.apache.org/jira/browse/CASSANDRA-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719269#comment-17719269 ] Ekaterina Dimitrova commented on CASSANDRA-18467: - [~jakubzytka] helped me to figure out the issue. The SDK had a few options for 11 and if it gets the wrong one which doesn't work, this triggers the IDE to assign default 17. It seems we might also want to add a step or at least instruction to ask people to remove .idea and .cassandra.iml before running *ant generate-idea-files* and pay attention to File -> Project Structure -> SDK With that said I tested that the currently proposed patch not only looks good to me but also works as expected. I triggered CI: [8+11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=CASSANDRA-18467], [11+17|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18467-JDK17] > Update generate-idea-files for J17 > -- > > Key: CASSANDRA-18467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18467 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Jakub Zytka >Priority: Low > Fix For: 5.x > > > There was a discussion in CASSANDRA-18258 how to update generate-idea-files. > The final agreement was to create one target to cover both Java 11 and Java > 17. > It will be good to figure out CASSANDRA-18263 and reshuffle arguments and > tasks based on what we decide to use as gc in testing for both Java 11 and > Java 17. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18474) Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
[
https://issues.apache.org/jira/browse/CASSANDRA-18474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719263#comment-17719263
]
Michael Semb Wever commented on CASSANDRA-18474:
trunk's CHANGES.txt fixed in
https://github.com/apache/cassandra/commit/5acfde21fef39dfc7cd2700323429ba9e982aac1
> Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
> ---
>
> Key: CASSANDRA-18474
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18474
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Repair, Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.10, 4.1.2, 5.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The {{SyncRequest}} message assumes the initiator/src/dst are all the IP
> address
> type as an optimization when calculating serialized size. It needs to handle
> mixed address families.
> {code}
> /1.2.3.4:7000->/[::1]:7000-SMALL_MESSAGES-f83ce0bc dropping message of type
> SYNC_REQ due to error"
> {code}
> {Exception}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 158, actual size at least 170, for verb SYNC_REQ
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18443) Deadlock updating sstable metadata if disk boundaries need reloading
[
https://issues.apache.org/jira/browse/CASSANDRA-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719264#comment-17719264
]
Michael Semb Wever commented on CASSANDRA-18443:
trunk's CHANGES.txt fixed in
https://github.com/apache/cassandra/commit/5acfde21fef39dfc7cd2700323429ba9e982aac1
> Deadlock updating sstable metadata if disk boundaries need reloading
>
>
> Key: CASSANDRA-18443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18443
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Local/Memtable, Local/SSTable
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.10, 4.1.2, 5.0
>
>
> {{CompactionStrategyManager.handleNotification}} holds the read lock while
> processing notifications. When handling metadata changed notifications, an
> extra call is made to maybeReloadDiskBoundaries which tries to grab the write
> lock and deadlocks the thread.
> Partial stacktrace
> {code}
> at jdk.internal.misc.Unsafe.park(java.base@11.0.16/Native Method)
> - parking to wait for <0x0005cc78> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire
> at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.maybeReloadDiskBoundaries(CompactionStrategyManager.java:495)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.getCompactionStrategyFor(CompactionStrategyManager.java:343)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleMetadataChangedNotification(CompactionStrategyManager.java:796)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(CompactionStrategyManager.java:838)
> at
> org.apache.cassandra.db.lifecycle.Tracker.notifySSTableMetadataChanged(Tracker.java:482)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(CompactionStrategyManager.java:838)
> {code}
> Deadlocking with the read lock held blocks the SlabpoolCleaner while
> notifying ColumnFamilyStore so memtables are prevented from being flushed and
> recycled, causing any thread applying a mutation to the database (at least
> GossipStage and MutationStage) to be considered down by peers and/or back up
> with pending requests.
> All the cases investigated were during single sstable upleveling by
> {{org.apache.cassandra.db.compaction.SingleSSTableLCSTask}} added in
> CASSANDRA-12526.
> Other less critical work was also affected, JMX calls to get estimated
> remaining compaction tasks, the index summary manager redistributing
> summaries, the StatusLogger trying to log dropped messages, and the
> ValidationManager.
> Workaround is to reboot the affected host.
> The fix is to just remove the redundant disk boundary reload check on that
> path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18260) Add details to Error message: Not enough space for compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-18260:
---
Fix Version/s: 4.0.10
4.1.2
5.0
(was: 5.x)
(was: 4.0.x)
(was: 4.1.x)
Source Control Link:
https://github.com/apache/cassandra/commit/65c99bfc42015ccfbcbc8963b3fb3c2a761ea053
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed as [65c99bfc42015ccfbcbc8963b3fb3c2a761ea053
|https://github.com/apache/cassandra/commit/65c99bfc42015ccfbcbc8963b3fb3c2a761ea053].
> Add details to Error message: Not enough space for compaction
> --
>
> Key: CASSANDRA-18260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Observability/Logging
>Reporter: Brad Schoening
>Assignee: Henrik Ingo
>Priority: Normal
> Fix For: 4.0.10, 4.1.2, 5.0
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> When compaction fails, the log message should list a) the free space
> available on disk at that point in time and b) perhaps the number and/or size
> of the source sstables being compacted.
> Free space can change from one moment to the next, so when the below
> compaction failed because it needed 161GB, upon checking the server a few
> minutes later, it had 184GB free. Similarly, the error message mentions it
> was writing out one SSTable on this STCS table, but its not clear if it was
> combining X -> 1 tables, or something else.
> {noformat}
> [INFO ] [CompactionExecutor:77758] cluster_id=87 ip_address=127.1.1.1
> CompactionTask.java:241 - Compacted (8a1cffe0-abb5-11ed-b3fc-8d2ac2c52f0d) 1
> sstables to [...] to level=0. 86.997GiB to 86.997GiB (~99% of original) in
> 1,508,155ms. Read Throughput = 59.069MiB/s, Write Throughput = 59.069MiB/s,
> Row Throughput = ~20,283/s. 21,375 total partitions merged to 21,375.
> Partition merge counts were \{1:21375, }
> [ERROR] [CompactionExecutor:4] cluster_id=87 ip_address=127.1.1.1
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:4,1,main]
> java.lang.RuntimeException: Not enough space for compaction, estimated
> sstables = 1, expected write size = 161228934093
> at
> org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:386)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:126)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.execute(CompactionManager.java:613)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:377)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16718) Changing listen_address with prefer_local may lead to issues
[ https://issues.apache.org/jira/browse/CASSANDRA-16718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719261#comment-17719261 ] Brandon Williams commented on CASSANDRA-16718: -- Branch links above updated so the commit you need is first; no code changes. > Changing listen_address with prefer_local may lead to issues > > > Key: CASSANDRA-16718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16718 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Jan Karlsson >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.x > > > Many container based solution function by assigning new listen_addresses when > nodes are stopped. Changing the listen_address is usually as simple as > turning off the node and changing the yaml file. > However, if prefer_local is enabled, I observed that nodes were unable to > join the cluster and fail with 'Unable to gossip with any seeds'. > Trace shows that the changing node will try to communicate with the existing > node but the response is never received. I assume it is because the existing > node attempts to communicate with the local address during the shadow round. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.1' into trunk
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 5acfde21fef39dfc7cd2700323429ba9e982aac1 Merge: ccf3789219 82dc54c3a8 Author: Mick Semb Wever AuthorDate: Thu May 4 12:34:53 2023 +0200 Merge branch 'cassandra-4.1' into trunk * cassandra-4.1: Improve 'Not enough space for compaction' logging messages CHANGES.txt| 7 +- src/java/org/apache/cassandra/db/Directories.java | 11 +- .../org/apache/cassandra/db/DirectoriesTest.java | 118 + 3 files changed, 131 insertions(+), 5 deletions(-) diff --cc CHANGES.txt index a6c69faef2,a775ce8ff3..862755286a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,139 -1,3 +1,137 @@@ +5.0 + * Replace usages of json-simple dependency by Jackson (CASSANDRA-16855) + * When decommissioning should set Severity to limit traffic (CASSANDRA-18430) + * For Java11 and Java17 remove -XX:-UseBiasedLocking as it is the default already (CASSANDRA-17869) + * Upgrade commons-io to 2.11.0 (CASSANDRA-17364) + * Node Draining Should Abort All Current SSTables Imports (CASSANDRA-18373) + * Use snake case for the names of CQL native functions (CASSANDRA-18037) + * Use jdk-dependent checkstyle version to check the source code (CASSANDRA-18262) + * Provide summary of failed SessionInfo's in StreamResultFuture (CASSANDRA-17199) + * CEP-20: Dynamic Data Masking (CASSANDRA-17940) + * Add system_views.snapshots virtual table (CASSANDRA-18102) + * Update OpenHFT dependencies (chronicle-queue, chronicle-core, chronicle-bytes, chronicle-wire, chronicle-threads) (CASSANDRA-18049) + * Remove org.apache.cassandra.hadoop code (CASSANDRA-18323) + * Remove deprecated CQL functions dateOf and unixTimestampOf (CASSANDRA-18328) + * Remove DateTieredCompactionStrategy (CASSANDRA-18043) + * Add system_views.max_sstable_size and system_views.max_sstable_duration tables (CASSANDRA-18333) + * Extend implicit allow-filtering for virtual tables to clustering columns (CASSANDRA-18331) + * Upgrade maven-shade-plugin to 3.4.1 to fix shaded dtest JAR build (CASSANDRA-18136) + * Upgrade to Opcodes.ASM9 (CASSANDRA-17971) + * Add MaxSSTableSize and MaxSSTableDuration metrics and propagate them together with local read/write ratio to tablestats (CASSANDRA-18283) + * Add more logging around CompactionManager operations (CASSANDRA-18268) + * Reduce memory allocations of calls to ByteBufer.duplicate() made in org.apache.cassandra.transport.CBUtil#writeValue (CASSANDRA-18212) + * CEP-17: SSTable API (CASSANDRA-17056) + * Gossip stateMapOrdering does not have correct ordering when both EndpointState are in the bootstrapping set (CASSANDRA-18292) + * Snapshot only sstables containing mismatching ranges on preview repair mismatch (CASSANDRA-17561) + * More accurate skipping of sstables in read path (CASSANDRA-18134) + * Prepare for JDK17 experimental support (CASSANDRA-18179, CASSANDRA-18258) + * Remove Scripted UDFs internals; hooks to be added later in CASSANDRA-17281 (CASSANDRA-18252) + * Update JNA to 5.13.0 (CASSANDRA-18050) + * Make virtual tables decide if they implicitly enable ALLOW FILTERING (CASSANDRA-18238) + * Add row, tombstone, and sstable count to nodetool profileload (CASSANDRA-18022) + * Coordinator level metrics for read response and mutation row and column counts (CASSANDRA-18155) + * Add CQL functions for dynamic data masking (CASSANDRA-17941) + * Print friendly error when nodetool attempts to connect to uninitialized server (CASSANDRA-11537) + * Use G1GC by default, and update default G1GC settings (CASSANDRA-18027) + * SimpleSeedProvider can resolve multiple IP addresses per DNS record (CASSANDRA-14361) + * Remove mocking in InternalNodeProbe spying on StorageServiceMBean (CASSANDRA-18152) + * Add compaction_properties column to system.compaction_history table and nodetool compactionhistory command (CASSANDRA-18061) + * Remove ProtocolVersion entirely from the CollectionSerializer ecosystem (CASSANDRA-18114) + * Fix serialization error in new getsstables --show-levels option (CASSANDRA-18140) + * Use checked casts when reading vints as ints (CASSANDRA-18099) + * Add Mutation Serialization Caching (CASSANDRA-17998) + * Only reload compaction strategies if disk boundaries change (CASSANDRA-17874) + * CEP-10: Simulator Java11 Support (CASSANDRA-17178) + * Set the major compaction type correctly for compactionstats (CASSANDRA-18055) + * Print exception message without stacktrace when nodetool commands fail on probe.getOwnershipWithPort() (CASSANDRA-18079) + * Add option to print level in nodetool getsstables output (CASSANDRA-18023) + * Implement a guardrail for not having zero default_time_to_live on tables with TWCS (CASSANDRA-18042) + * Add CQL scalar functions for collection aggregation (CASSANDRA-18060) + * Make cassandra.replayList
[cassandra] branch cassandra-4.1 updated (602ffcbf3e -> 82dc54c3a8)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 602ffcbf3e fix flaky o.a.c.distributed.test.PaxosRepair2Test.paxosRepairHistoryIsntUpdatedInForcedRepair new 65c99bfc42 Improve 'Not enough space for compaction' logging messages new 82dc54c3a8 Merge branch 'cassandra-4.0' into cassandra-4.1 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + src/java/org/apache/cassandra/db/Directories.java | 23 ++- .../org/apache/cassandra/db/DirectoriesTest.java | 179 + 3 files changed, 202 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (ccf3789219 -> 5acfde21fe)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from ccf3789219 Merge branch 'cassandra-4.1' into trunk new 65c99bfc42 Improve 'Not enough space for compaction' logging messages new 82dc54c3a8 Merge branch 'cassandra-4.0' into cassandra-4.1 new 5acfde21fe Merge branch 'cassandra-4.1' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 7 +- src/java/org/apache/cassandra/db/Directories.java | 11 +- .../org/apache/cassandra/db/DirectoriesTest.java | 118 + 3 files changed, 131 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.0' into cassandra-4.1
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-4.1
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 82dc54c3a89ed55e14b04a0d99dc59782991f99e
Merge: 602ffcbf3e 65c99bfc42
Author: Mick Semb Wever
AuthorDate: Thu May 4 12:24:47 2023 +0200
Merge branch 'cassandra-4.0' into cassandra-4.1
* cassandra-4.0:
Improve 'Not enough space for compaction' logging messages
CHANGES.txt| 1 +
src/java/org/apache/cassandra/db/Directories.java | 23 ++-
.../org/apache/cassandra/db/DirectoriesTest.java | 179 +
3 files changed, 202 insertions(+), 1 deletion(-)
diff --cc CHANGES.txt
index e6af024cd3,b3f3cf883a..a775ce8ff3
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,11 -1,5 +1,12 @@@
-4.0.10
+4.1.2
+ * Allow keystore and trustrore passwords to be nullable (CASSANDRA-18124)
+ * Return snapshots with dots in their name in nodetool listsnapshots
(CASSANDRA-18371)
+ * Fix NPE when loading snapshots and data directory is one directory from
root (CASSANDRA-18359)
+ * Do not submit hints when hinted_handoff_enabled=false (CASSANDRA-18304)
+ * Fix COPY ... TO STDOUT behavior in cqlsh (CASSANDRA-18353)
+ * Remove six and Py2SaferScanner merge cruft (CASSANDRA-18354)
+Merged from 4.0:
+ * Improve 'Not enough space for compaction' logging messages
(CASSANDRA-18260)
* Incremental repairs fail on mixed IPv4/v6 addresses serializing
SyncRequest (CASSANDRA-18474)
* Deadlock updating sstable metadata if disk boundaries need reloading
(CASSANDRA-18443)
* Fix nested selection of reversed collections (CASSANDRA-17913)
diff --cc src/java/org/apache/cassandra/db/Directories.java
index 9f80ce0619,cecc60d75c..b16dd972f1
--- a/src/java/org/apache/cassandra/db/Directories.java
+++ b/src/java/org/apache/cassandra/db/Directories.java
@@@ -499,11 -492,32 +499,32 @@@ public class Directorie
continue;
DataDirectoryCandidate candidate = new
DataDirectoryCandidate(dataDir);
// exclude directory if its total writeSize does not fit to data
directory
+ logger.debug("DataDirectory {} has {} bytes available, checking
if we can write {} bytes", dataDir.location, candidate.availableSpace,
writeSize);
if (candidate.availableSpace < writeSize)
+ {
+ logger.warn("DataDirectory {} can't be used for compaction.
Only {} is available, but {} is the minimum write size.",
+ candidate.dataDirectory.location,
+
FileUtils.stringifyFileSize(candidate.availableSpace),
+ FileUtils.stringifyFileSize(writeSize));
continue;
+ }
totalAvailable += candidate.availableSpace;
}
- return totalAvailable > expectedTotalWriteSize;
+
+ if (totalAvailable <= expectedTotalWriteSize)
+ {
+ StringJoiner pathString = new StringJoiner(",", "[", "]");
+ for (DataDirectory p: paths)
+ {
-pathString.add(p.location.getAbsolutePath());
++pathString.add(p.location.toJavaIOFile().getAbsolutePath());
+ }
+ logger.warn("Insufficient disk space for compaction. Across {}
there's only {} available, but {} is needed.",
+ pathString.toString(),
+ FileUtils.stringifyFileSize(totalAvailable),
+ FileUtils.stringifyFileSize(expectedTotalWriteSize));
+ return false;
+ }
+ return true;
}
public DataDirectory[] getWriteableLocations()
diff --cc test/unit/org/apache/cassandra/db/DirectoriesTest.java
index 252c630614,022c3a07d5..c701516d1e
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@@ -20,19 -20,10 +20,20 @@@ package org.apache.cassandra.db
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
-import java.util.*;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.IdentityHashMap;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+ import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.Callable;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
@@@ -46,10 -36,22 +49,20 @@@ import org.junit.AfterClass
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.apache.cassandra.Util;
+ import org.slf4j.LoggerFactory;
+ import or
[cassandra] branch cassandra-4.0 updated: Improve 'Not enough space for compaction' logging messages
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-4.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-4.0 by this push:
new 65c99bfc42 Improve 'Not enough space for compaction' logging messages
65c99bfc42 is described below
commit 65c99bfc42015ccfbcbc8963b3fb3c2a761ea053
Author: Henrik Ingo
AuthorDate: Wed Apr 19 17:34:23 2023 +0300
Improve 'Not enough space for compaction' logging messages
Adds separate WARN level messages to the check itself in Directories.
patch by Henrik Ingo; reviewed by Mick Semb Wever, Brad Schoening, Claude
Warren, Kan Maung, maxwellguo for CASSANDRA-18260
---
CHANGES.txt| 1 +
src/java/org/apache/cassandra/db/Directories.java | 23 ++-
.../org/apache/cassandra/db/DirectoriesTest.java | 177 +
3 files changed, 200 insertions(+), 1 deletion(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index f2d7040a1e..b3f3cf883a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0.10
+ * Improve 'Not enough space for compaction' logging messages (CASSANDRA-18260)
* Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
(CASSANDRA-18474)
* Deadlock updating sstable metadata if disk boundaries need reloading
(CASSANDRA-18443)
* Fix nested selection of reversed collections (CASSANDRA-17913)
diff --git a/src/java/org/apache/cassandra/db/Directories.java
b/src/java/org/apache/cassandra/db/Directories.java
index cf770e6375..cecc60d75c 100644
--- a/src/java/org/apache/cassandra/db/Directories.java
+++ b/src/java/org/apache/cassandra/db/Directories.java
@@ -492,11 +492,32 @@ public class Directories
continue;
DataDirectoryCandidate candidate = new
DataDirectoryCandidate(dataDir);
// exclude directory if its total writeSize does not fit to data
directory
+logger.debug("DataDirectory {} has {} bytes available, checking if
we can write {} bytes", dataDir.location, candidate.availableSpace, writeSize);
if (candidate.availableSpace < writeSize)
+{
+logger.warn("DataDirectory {} can't be used for compaction.
Only {} is available, but {} is the minimum write size.",
+candidate.dataDirectory.location,
+
FileUtils.stringifyFileSize(candidate.availableSpace),
+FileUtils.stringifyFileSize(writeSize));
continue;
+}
totalAvailable += candidate.availableSpace;
}
-return totalAvailable > expectedTotalWriteSize;
+
+if (totalAvailable <= expectedTotalWriteSize)
+{
+StringJoiner pathString = new StringJoiner(",", "[", "]");
+for (DataDirectory p: paths)
+{
+pathString.add(p.location.getAbsolutePath());
+}
+logger.warn("Insufficient disk space for compaction. Across {}
there's only {} available, but {} is needed.",
+pathString.toString(),
+FileUtils.stringifyFileSize(totalAvailable),
+FileUtils.stringifyFileSize(expectedTotalWriteSize));
+return false;
+}
+return true;
}
public DataDirectory[] getWriteableLocations()
diff --git a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
index 2ba8952886..022c3a07d5 100644
--- a/test/unit/org/apache/cassandra/db/DirectoriesTest.java
+++ b/test/unit/org/apache/cassandra/db/DirectoriesTest.java
@@ -23,6 +23,7 @@ import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.*;
+import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.Callable;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
@@ -30,11 +31,22 @@ import java.util.concurrent.Future;
import com.google.common.collect.Sets;
import org.apache.commons.lang3.StringUtils;
+import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
+import org.slf4j.LoggerFactory;
+import org.slf4j.MDC;
+// Our version of Sfl4j seems to be missing the ListAppender class.
+// Future sfl4j versions have one. At that time the below imports can be
+// replaced with `org.slf4j.*` equivalents.
+import ch.qos.logback.classic.Level;
+import ch.qos.logback.classic.Logger;
+import ch.qos.logback.classic.spi.ILoggingEvent;
+import ch.qos.logback.core.read.ListAppender;
+
import org.apache.cassandra.cql3.ColumnIdentifier;
import org.apache.cassandra.schema.Indexes;
import org.apache.cassandra.schema.SchemaConstants;
@@ -71,6 +83,13 @@ public class DirectoriesTest
private static Set CFM;
private static Ma
[jira] [Commented] (CASSANDRA-16718) Changing listen_address with prefer_local may lead to issues
[ https://issues.apache.org/jira/browse/CASSANDRA-16718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719257#comment-17719257 ] Brandon Williams commented on CASSANDRA-16718: -- Heh, it's actually in there, it's just the code is from 2021 at this point. Let me recreate the branches so it's easy to find. > Changing listen_address with prefer_local may lead to issues > > > Key: CASSANDRA-16718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16718 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Jan Karlsson >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.x > > > Many container based solution function by assigning new listen_addresses when > nodes are stopped. Changing the listen_address is usually as simple as > turning off the node and changing the yaml file. > However, if prefer_local is enabled, I observed that nodes were unable to > join the cluster and fail with 'Unable to gossip with any seeds'. > Trace shows that the changing node will try to communicate with the existing > node but the response is never received. I assume it is because the existing > node attempts to communicate with the local address during the shadow round. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18228) Reorganize Apache C* docs in trunk branch to match the IA
[ https://issues.apache.org/jira/browse/CASSANDRA-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719256#comment-17719256 ] Stefan Miklosovic commented on CASSANDRA-18228: --- [~polandll] would you please check what is going on here? I would like to contribute more to docs, fixes of typos etc but I am quite puzzled what is going on here. > Reorganize Apache C* docs in trunk branch to match the IA > - > > Key: CASSANDRA-18228 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18228 > Project: Cassandra > Issue Type: New Feature > Components: Documentation >Reporter: Lorina Poland >Assignee: Lorina Poland >Priority: Normal > Fix For: 5.0 > > > An agreed-upon Information Architecture will be used to reorganize the C* 5.0 > docs. > IA doc: > https://docs.google.com/document/d/1A96K73vj9MbJoD7wJNgIKWrOkLq-ZL2cNZAEXSWrciY/edit?usp=sharing -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18260) Add details to Error message: Not enough space for compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-18260:
---
Status: Review In Progress (was: Needs Committer)
> Add details to Error message: Not enough space for compaction
> --
>
> Key: CASSANDRA-18260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Observability/Logging
>Reporter: Brad Schoening
>Assignee: Henrik Ingo
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> When compaction fails, the log message should list a) the free space
> available on disk at that point in time and b) perhaps the number and/or size
> of the source sstables being compacted.
> Free space can change from one moment to the next, so when the below
> compaction failed because it needed 161GB, upon checking the server a few
> minutes later, it had 184GB free. Similarly, the error message mentions it
> was writing out one SSTable on this STCS table, but its not clear if it was
> combining X -> 1 tables, or something else.
> {noformat}
> [INFO ] [CompactionExecutor:77758] cluster_id=87 ip_address=127.1.1.1
> CompactionTask.java:241 - Compacted (8a1cffe0-abb5-11ed-b3fc-8d2ac2c52f0d) 1
> sstables to [...] to level=0. 86.997GiB to 86.997GiB (~99% of original) in
> 1,508,155ms. Read Throughput = 59.069MiB/s, Write Throughput = 59.069MiB/s,
> Row Throughput = ~20,283/s. 21,375 total partitions merged to 21,375.
> Partition merge counts were \{1:21375, }
> [ERROR] [CompactionExecutor:4] cluster_id=87 ip_address=127.1.1.1
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:4,1,main]
> java.lang.RuntimeException: Not enough space for compaction, estimated
> sstables = 1, expected write size = 161228934093
> at
> org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:386)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:126)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.execute(CompactionManager.java:613)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:377)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18260) Add details to Error message: Not enough space for compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-18260:
---
Status: Ready to Commit (was: Review In Progress)
> Add details to Error message: Not enough space for compaction
> --
>
> Key: CASSANDRA-18260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Observability/Logging
>Reporter: Brad Schoening
>Assignee: Henrik Ingo
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> When compaction fails, the log message should list a) the free space
> available on disk at that point in time and b) perhaps the number and/or size
> of the source sstables being compacted.
> Free space can change from one moment to the next, so when the below
> compaction failed because it needed 161GB, upon checking the server a few
> minutes later, it had 184GB free. Similarly, the error message mentions it
> was writing out one SSTable on this STCS table, but its not clear if it was
> combining X -> 1 tables, or something else.
> {noformat}
> [INFO ] [CompactionExecutor:77758] cluster_id=87 ip_address=127.1.1.1
> CompactionTask.java:241 - Compacted (8a1cffe0-abb5-11ed-b3fc-8d2ac2c52f0d) 1
> sstables to [...] to level=0. 86.997GiB to 86.997GiB (~99% of original) in
> 1,508,155ms. Read Throughput = 59.069MiB/s, Write Throughput = 59.069MiB/s,
> Row Throughput = ~20,283/s. 21,375 total partitions merged to 21,375.
> Partition merge counts were \{1:21375, }
> [ERROR] [CompactionExecutor:4] cluster_id=87 ip_address=127.1.1.1
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:4,1,main]
> java.lang.RuntimeException: Not enough space for compaction, estimated
> sstables = 1, expected write size = 161228934093
> at
> org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:386)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:126)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.execute(CompactionManager.java:613)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:377)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-12937) Default setting (yaml) for SSTable compression
[
https://issues.apache.org/jira/browse/CASSANDRA-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719227#comment-17719227
]
Stefan Miklosovic edited comment on CASSANDRA-12937 at 5/4/23 9:23 AM:
---
I did another round of review:
When I do this
{code:java}
commitlog_compression:
- class_name: lz4
parameters:
- enable: "true"
lz4_compressor_type: "fast" {code}
This does not work:
{code:java}
Caused by: org.apache.cassandra.exceptions.ConfigurationException: Could not
create Compression for type org.apache.cassandra.io.compress.lz4
at
org.apache.cassandra.schema.CompressionParams.parseCompressorClass(CompressionParams.java:338)
at
org.apache.cassandra.schema.CompressionParams.createCompressor(CompressionParams.java:396)
at
org.apache.cassandra.db.commitlog.CommitLog$Configuration.(CommitLog.java:630)
at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:107)
at org.apache.cassandra.db.commitlog.CommitLog.construct(CommitLog.java:92)
at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:77)
{code}
I would expect similar errors to be found when trying aliases for other
_compression configuration properties.
Next, the default configuration for sstable_compression which is currently this
does not work:
{code:java}
sstable_compression:
# - class_name: lz4
# parameters:
# - enable: true
# chunk_length: 16KiB
# min_compress_ratio: 0.0
# max_comrpessed_length: 16KiB
# class_specific_parameter: value {code}
When I uncomment it like this:
{code:java}
sstable_compression:
- class_name: lz4
parameters:
- enable: true
chunk_length: 16KiB
min_compress_ratio: 0.0
max_comrpessed_length: 16KiB
class_specific_parameter: value {code}
First of all it says that:
{code:java}
Caused by: java.lang.ClassCastException: java.lang.Boolean cannot be cast to
java.lang.CharSequence
at
org.apache.cassandra.schema.CompressionParams.copyOptions(CompressionParams.java:406)
at
org.apache.cassandra.schema.CompressionParams.fromParameterizedClass(CompressionParams.java:160)
at
org.apache.cassandra.schema.CompressionParams.defaultParams(CompressionParams.java:150)
at
org.apache.cassandra.schema.TableParams$Builder.(TableParams.java:353)
at org.apache.cassandra.schema.TableParams.builder(TableParams.java:119)
{code}
This means that it expects values to be strings, so I do it like this:
{code:java}
sstable_compression:
- class_name: lz4
parameters:
- enable: "true"
chunk_length: "16KiB"
min_compress_ratio: "0.0"
max_comrpessed_length: "16KiB"
class_specific_parameter: "value" {code}
But it says:
{code:java}
Caused by: org.apache.cassandra.exceptions.ConfigurationException: Unknown
compression options enable
at
org.apache.cassandra.schema.CompressionParams.createCompressor(CompressionParams.java:358)
at
org.apache.cassandra.schema.CompressionParams.(CompressionParams.java:260)
at
org.apache.cassandra.io.compress.CompressionMetadata.open(CompressionMetadata.java:94)
... 15 common frames omitted {code}
I think this has to be changed to "enabled" as it is "enabled" in trunk too.
Next, it does not seem to me that the value of "lz4" is picked up because if I
do this:
{code:java}
sstable_compression:
- class_name: lz4
parameters:
- enabled: "true"
chunk_length: "16KiB"
min_compress_ratio: "0.0"
max_comrpessed_length: "16KiB"
class_specific_parameter: "value"
{code}
it does not say that the "class_specific_parameter" is invalid, check this:
when I do
{code:java}
sstable_compression:
- class_name: LZ4Compressor
parameters:
- enabled: "true"
chunk_length: "16KiB"
min_compress_ratio: "0.0"
max_comrpessed_length: "16KiB"
class_specific_parameter: "value" {code}
it throws:
{code:java}
Caused by: org.apache.cassandra.exceptions.ConfigurationException: Unknown
compression options class_specific_parameter
at
org.apache.cassandra.schema.CompressionParams.createCompressor(CompressionParams.java:358)
at
org.apache.cassandra.schema.CompressionParams.lambda$fromClassAndOptions$0(CompressionParams.java:228)
at
org.apache.cassandra.schema.CompressionParams.fromClassAndOptions(CompressionParams.java:229)
at
org.apache.cassandra.schema.CompressionParams.fromParameterizedClass(CompressionParams.java:160)
at
org.apache.cassandra.schema.CompressionParams.defaultParams(CompressionParams.java:150)
at
org.apache.cassandra.schema.TableParams$Builder.(TableParams.java:353)
at org.apache.cassandra.schema.TableParams.builder(TableParams.java:119)
at
org.apache.cassandra.cql3.statements.schema.TableAttributes.validate(TableAttributes.java:60)
at
org.apac
[jira] [Commented] (CASSANDRA-12937) Default setting (yaml) for SSTable compression
[
https://issues.apache.org/jira/browse/CASSANDRA-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719227#comment-17719227
]
Stefan Miklosovic commented on CASSANDRA-12937:
---
I did another round of review:
When I do this
{code:java}
commitlog_compression:
- class_name: lz4
parameters:
- enable: "true"
lz4_compressor_type: "fast" {code}
This does not work:
{code:java}
Caused by: org.apache.cassandra.exceptions.ConfigurationException: Could not
create Compression for type org.apache.cassandra.io.compress.lz4
at
org.apache.cassandra.schema.CompressionParams.parseCompressorClass(CompressionParams.java:338)
at
org.apache.cassandra.schema.CompressionParams.createCompressor(CompressionParams.java:396)
at
org.apache.cassandra.db.commitlog.CommitLog$Configuration.(CommitLog.java:630)
at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:107)
at org.apache.cassandra.db.commitlog.CommitLog.construct(CommitLog.java:92)
at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:77)
{code}
I would expect similar errors to be found when trying aliases for other
_compression configuration properties.
Next, the default configuration for sstable_compression which is currently this
does not work:
{code:java}
sstable_compression:
# - class_name: lz4
# parameters:
# - enable: true
# chunk_length: 16KiB
# min_compress_ratio: 0.0
# max_comrpessed_length: 16KiB
# class_specific_parameter: value {code}
When I uncomment it like this:
{code:java}
sstable_compression:
- class_name: lz4
parameters:
- enable: true
chunk_length: 16KiB
min_compress_ratio: 0.0
max_comrpessed_length: 16KiB
class_specific_parameter: value {code}
First of all it says that:
{code:java}
Caused by: java.lang.ClassCastException: java.lang.Boolean cannot be cast to
java.lang.CharSequence
at
org.apache.cassandra.schema.CompressionParams.copyOptions(CompressionParams.java:406)
at
org.apache.cassandra.schema.CompressionParams.fromParameterizedClass(CompressionParams.java:160)
at
org.apache.cassandra.schema.CompressionParams.defaultParams(CompressionParams.java:150)
at
org.apache.cassandra.schema.TableParams$Builder.(TableParams.java:353)
at org.apache.cassandra.schema.TableParams.builder(TableParams.java:119)
{code}
This means that it expects values to be strings, so I do it like this:
{code:java}
sstable_compression:
- class_name: lz4
parameters:
- enable: "true"
chunk_length: "16KiB"
min_compress_ratio: "0.0"
max_comrpessed_length: "16KiB"
class_specific_parameter: "value" {code}
But it says:
{code:java}
Caused by: org.apache.cassandra.exceptions.ConfigurationException: Unknown
compression options enable
at
org.apache.cassandra.schema.CompressionParams.createCompressor(CompressionParams.java:358)
at
org.apache.cassandra.schema.CompressionParams.(CompressionParams.java:260)
at
org.apache.cassandra.io.compress.CompressionMetadata.open(CompressionMetadata.java:94)
... 15 common frames omitted {code}
I think this has to be changed to "enabled" as it is "enabled" in trunk too.
Next, it does not seem to me that the value of "lz4" is picked up because if I
do this:
{code:java}
sstable_compression:
- class_name: lz4
parameters:
- enabled: "true"
chunk_length: "16KiB"
min_compress_ratio: "0.0"
max_comrpessed_length: "16KiB"
class_specific_parameter: "value"
{code}
it does not say that the "class_specific_parameter" is invalid, check this:
when I do
{code:java}
sstable_compression:
- class_name: LZ4Compressor
parameters:
- enabled: "true"
chunk_length: "16KiB"
min_compress_ratio: "0.0"
max_comrpessed_length: "16KiB"
class_specific_parameter: "value" {code}
it throws:
{code:java}
Caused by: org.apache.cassandra.exceptions.ConfigurationException: Unknown
compression options class_specific_parameter
at
org.apache.cassandra.schema.CompressionParams.createCompressor(CompressionParams.java:358)
at
org.apache.cassandra.schema.CompressionParams.lambda$fromClassAndOptions$0(CompressionParams.java:228)
at
org.apache.cassandra.schema.CompressionParams.fromClassAndOptions(CompressionParams.java:229)
at
org.apache.cassandra.schema.CompressionParams.fromParameterizedClass(CompressionParams.java:160)
at
org.apache.cassandra.schema.CompressionParams.defaultParams(CompressionParams.java:150)
at
org.apache.cassandra.schema.TableParams$Builder.(TableParams.java:353)
at org.apache.cassandra.schema.TableParams.builder(TableParams.java:119)
at
org.apache.cassandra.cql3.statements.schema.TableAttributes.validate(TableAttributes.java:60)
at
org.apache.cassandra.cql3.statements.schema.CreateTable
[jira] [Updated] (CASSANDRA-12937) Default setting (yaml) for SSTable compression
[
https://issues.apache.org/jira/browse/CASSANDRA-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-12937:
--
Status: Changes Suggested (was: Review In Progress)
> Default setting (yaml) for SSTable compression
> --
>
> Key: CASSANDRA-12937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12937
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Michael Semb Wever
>Assignee: Claude Warren
>Priority: Low
> Labels: AdventCalendar2021, lhf
> Fix For: 5.x
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> In many situations the choice of compression for sstables is more relevant to
> the disks attached than to the schema and data.
> This issue is to add to cassandra.yaml a default value for sstable
> compression that new tables will inherit (instead of the defaults found in
> {{CompressionParams.DEFAULT}}.
> Examples where this can be relevant are filesystems that do on-the-fly
> compression (btrfs, zfs) or specific disk configurations or even specific C*
> versions (see CASSANDRA-10995 ).
> +Additional information for newcomers+
> Some new fields need to be added to {{cassandra.yaml}} to allow specifying
> the field required for defining the default compression parameters. In
> {{DatabaseDescriptor}} a new {{CompressionParams}} field should be added for
> the default compression. This field should be initialized in
> {{DatabaseDescriptor.applySimpleConfig()}}. At the different places where
> {{CompressionParams.DEFAULT}} was used the code should call
> {{DatabaseDescriptor#getDefaultCompressionParams}} that should return some
> copy of configured {{CompressionParams}}.
> Some unit test using {{OverrideConfigurationLoader}} should be used to test
> that the table schema use the new default when a new table is created (see
> CreateTest for some example).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18485) CEP-15: (C*) Enhance in-memory FileSystem to work with mmap and support tests to add custom logic
[ https://issues.apache.org/jira/browse/CASSANDRA-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719215#comment-17719215 ] Berenguer Blasi commented on CASSANDRA-18485: - So [~dcapwell] having had a cursory look: - You mainly change the code to have a pluggable FS - You provide a FS implementation for a test of yours you needed Anything else I am missing? Asking bc my simulator knowledge is 0 I want to make sure the above is the surface area of the review. > CEP-15: (C*) Enhance in-memory FileSystem to work with mmap and support tests > to add custom logic > - > > Key: CASSANDRA-18485 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18485 > Project: Cassandra > Issue Type: Improvement > Components: Accord, Test/dtest/java, Test/unit >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 5.x > > Time Spent: 1h 50m > Remaining Estimate: 0h > > The Simulator currently uses JimFS for its FileSystem, but this lacks any way > to add custom logic when file operations are performed; it also lacks mmap > support which requires all disk access logic to provide non-mmap solutions as > well. > As part of the Simulator work, testing disk corruption will require resolving > both these issues, so will need a new FileSystem to rule them all… > This ticket is to define the new FileSystem and add the integration to > Simulator, JVM-Dtest, and Unit tests, but does not directly add the fault > injections that Simulator will be doing, that will be follow up work. > Goals: > * FileSystem that works for unit, jvm-dtest, and simulator tests > * FileSystem that allows tests to intercept file operations -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18495) Warnings when using the perl driver to connect to Cassandra
[
https://issues.apache.org/jira/browse/CASSANDRA-18495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719203#comment-17719203
]
Aswin Karthik edited comment on CASSANDRA-18495 at 5/4/23 8:22 AM:
---
Thanks for the response. Not sure if it is a duplicate. This does not involve
any creation of tables/keyspaces. It is about the recommended way to initialize
a session on a driver.
>From a driver's perspective, what is the right way to choose a keyspace on a
>connection?
was (Author: JIRAUSER289033):
Thanks for the response. Not sure if it is a duplicate. This does involve any
creation of tables/keyspaces. It is about the recommended way to initialize a
session on a driver.
>From a driver's perspective, what is the right way to choose a keyspace on a
>connection?
> Warnings when using the perl driver to connect to Cassandra
> ---
>
> Key: CASSANDRA-18495
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18495
> Project: Cassandra
> Issue Type: Bug
>Reporter: Aswin Karthik
>Priority: Normal
>
> When I use the [perl driver|https://github.com/TvdW/perl-DBD-Cassandra] to
> connect to Cassandra 4.0.3 and onwards, I get the following error just for
> initialization of a connection.
> The error I get is
> {noformat}
> `USE ` with prepared statements is considered to be an anti-pattern
> due to ambiguity in non-qualified table names. Please consider removing
> instances of `Session#setKeyspace()`, `Session#execute("USE
> ")` and `cluster.newSession()` from your code, and always
> use fully qualified table names (e.g. .). Keyspace used:
> null, statement keyspace: null, statement id: at
> Cassandra/Client/Connection.pm line 957.
> {noformat}
>
> This is just from initialization of the perl driver connection while choosing
> the keyspace.
> [https://github.com/TvdW/perl-DBD-Cassandra/blob/master/Cassandra-Client/lib/Cassandra/Client/Connection.pm#L562]
> before running any queries. It is emitted even if I use qualified prepared
> statement.
> The same warning does not pop up in Datastax java driver initialization.
> On debugging, I found that this warning does not emit for all unqualified
> prepared statements. It only emits for unqualified prepared "USE ks"
> statement. But the "USE ks" can never be qualified. So the warning is a bit
> vague on what is the recommended approach.
> And from the perspective of the driver, it is setting the keyspace of the
> connection for the first time. The same warning does not happen on the
> datastax java driver and that is because it uses QUERY to set the keyspace on
> connection. (I tried to follow the same approach on the perl driver -
> https://github.com/TvdW/perl-DBD-Cassandra/pull/35 )
> The warnings are not very clear on what is deprecated and what is not. Does
> it deprecate only the use of prepared statement of "USE ks"? or does it
> deprecate "USE ks" completely? And it is not being emitted for other
> unqualified prepared statements but only for a USE statement which cannot be
> qualified at all.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18495) Warnings when using the perl driver to connect to Cassandra
[
https://issues.apache.org/jira/browse/CASSANDRA-18495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719203#comment-17719203
]
Aswin Karthik commented on CASSANDRA-18495:
---
Thanks for the response. Not sure if it is a duplicate. This does involve any
creation of tables/keyspaces. It is about the recommended way to initialize a
session on a driver.
>From a driver's perspective, what is the right way to choose a keyspace on a
>connection?
> Warnings when using the perl driver to connect to Cassandra
> ---
>
> Key: CASSANDRA-18495
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18495
> Project: Cassandra
> Issue Type: Bug
>Reporter: Aswin Karthik
>Priority: Normal
>
> When I use the [perl driver|https://github.com/TvdW/perl-DBD-Cassandra] to
> connect to Cassandra 4.0.3 and onwards, I get the following error just for
> initialization of a connection.
> The error I get is
> {noformat}
> `USE ` with prepared statements is considered to be an anti-pattern
> due to ambiguity in non-qualified table names. Please consider removing
> instances of `Session#setKeyspace()`, `Session#execute("USE
> ")` and `cluster.newSession()` from your code, and always
> use fully qualified table names (e.g. .). Keyspace used:
> null, statement keyspace: null, statement id: at
> Cassandra/Client/Connection.pm line 957.
> {noformat}
>
> This is just from initialization of the perl driver connection while choosing
> the keyspace.
> [https://github.com/TvdW/perl-DBD-Cassandra/blob/master/Cassandra-Client/lib/Cassandra/Client/Connection.pm#L562]
> before running any queries. It is emitted even if I use qualified prepared
> statement.
> The same warning does not pop up in Datastax java driver initialization.
> On debugging, I found that this warning does not emit for all unqualified
> prepared statements. It only emits for unqualified prepared "USE ks"
> statement. But the "USE ks" can never be qualified. So the warning is a bit
> vague on what is the recommended approach.
> And from the perspective of the driver, it is setting the keyspace of the
> connection for the first time. The same warning does not happen on the
> datastax java driver and that is because it uses QUERY to set the keyspace on
> connection. (I tried to follow the same approach on the perl driver -
> https://github.com/TvdW/perl-DBD-Cassandra/pull/35 )
> The warnings are not very clear on what is deprecated and what is not. Does
> it deprecate only the use of prepared statement of "USE ks"? or does it
> deprecate "USE ks" completely? And it is not being emitted for other
> unqualified prepared statements but only for a USE statement which cannot be
> qualified at all.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (663290a00 -> 664c200d3)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 663290a00 generate docs for 52f24b20 new 664c200d3 generate docs for 52f24b20 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (663290a00) \ N -- N -- N refs/heads/asf-staging (664c200d3) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4796900 -> 4796900 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
