[jira] [Updated] (CASSANDRA-19185) Vector search tests are failing on recall accuracy
[ https://issues.apache.org/jira/browse/CASSANDRA-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19185: - Fix Version/s: 5.0-beta2 (was: 5.x) (was: 5.0.x) Since Version: 5.0-alpha1 Source Control Link: https://github.com/apache/cassandra/commit/7aab61b06357ce0b59977715f82fed1ad24474b4 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as https://github.com/apache/cassandra/commit/7aab61b06357ce0b59977715f82fed1ad24474b4 > Vector search tests are failing on recall accuracy > -- > > Key: CASSANDRA-19185 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19185 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta2 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Vector tests are failing randomly because they do not meet recall assertion > values. Currently, the following tests have been reported as failing: > VectorSegmentationTest.testMultipleSegmentsForCompaction > VectorDistributedTest.rangeRestrictedTest > VectorDistributedTest.testPartitionRestrictedVectorSearch > Since the vector searches are approximate and the vectors used in the tests > are random, it is unlikely that they will always meet a high recall. The > recall assertions are looking for recall values of 0.9 and above. Part of > this issue is related to the use of random values in the vectors being > tested. We have seen, with other tests, that the vector search performs > better with non-random generated datasets like the Glove datasets. As such, > there are the following available to fix these tests. > # Downgrade the assertions to a value that is likely to always pass. The > problem is that there is no guarantee that a test will always pass any recall > value we give it. > # Use generated datasets for these tests to see if that improves the recall > results. > # Remove the recall assertions unless they are specifically asked for. We > could use a system property to enable recall testing for targeted vector > testing. > I don't think option 1 is a viable long-term solution as we can never be > certain that it will always work. Option 2 has more promise but it could > still result in failures because of the approximate nature of the vector > searches. As such, option 3 seems the only viable solution here but means > that, in most cases, we are only really testing that we are returning results > from the search, not how accurate those results are. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19185) Vector search tests are failing on recall accuracy
[ https://issues.apache.org/jira/browse/CASSANDRA-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828055#comment-17828055 ] Mike Adamson commented on CASSANDRA-19185: -- Thanks, I've done a few commits now so should be good, but thanks for the offer. I'll get trunk propagated and tested and commit if good. > Vector search tests are failing on recall accuracy > -- > > Key: CASSANDRA-19185 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19185 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0.x, 5.x > > Time Spent: 0.5h > Remaining Estimate: 0h > > Vector tests are failing randomly because they do not meet recall assertion > values. Currently, the following tests have been reported as failing: > VectorSegmentationTest.testMultipleSegmentsForCompaction > VectorDistributedTest.rangeRestrictedTest > VectorDistributedTest.testPartitionRestrictedVectorSearch > Since the vector searches are approximate and the vectors used in the tests > are random, it is unlikely that they will always meet a high recall. The > recall assertions are looking for recall values of 0.9 and above. Part of > this issue is related to the use of random values in the vectors being > tested. We have seen, with other tests, that the vector search performs > better with non-random generated datasets like the Glove datasets. As such, > there are the following available to fix these tests. > # Downgrade the assertions to a value that is likely to always pass. The > problem is that there is no guarantee that a test will always pass any recall > value we give it. > # Use generated datasets for these tests to see if that improves the recall > results. > # Remove the recall assertions unless they are specifically asked for. We > could use a system property to enable recall testing for targeted vector > testing. > I don't think option 1 is a viable long-term solution as we can never be > certain that it will always work. Option 2 has more promise but it could > still result in failures because of the approximate nature of the vector > searches. As such, option 3 seems the only viable solution here but means > that, in most cases, we are only really testing that we are returning results > from the search, not how accurate those results are. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18940) SAI post-filtering reads don't update local table latency metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18940: - Reviewers: Mike Adamson > SAI post-filtering reads don't update local table latency metrics > - > > Key: CASSANDRA-18940 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18940 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index, Feature/SAI, Observability/Metrics >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: > draft_fix_for_SAI_post-filtering_reads_not_updating_local_table_metrics.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Once an SAI index finds matches (primary keys), it reads the associated rows > and post-filters them to incorporate partial writes, tombstones, etc. > However, those reads are not currently updating the local table latency > metrics. It should be simple enough to attach a metrics recording > transformation to the iterator produced by querying local storage. (I've > attached a patch that should apply cleanly to trunk, but there may be a > better way...) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18940) SAI post-filtering reads don't update local table latency metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson reassigned CASSANDRA-18940: Assignee: (was: Mike Adamson) > SAI post-filtering reads don't update local table latency metrics > - > > Key: CASSANDRA-18940 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18940 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index, Feature/SAI, Observability/Metrics >Reporter: Caleb Rackliffe >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: > draft_fix_for_SAI_post-filtering_reads_not_updating_local_table_metrics.patch > > > Once an SAI index finds matches (primary keys), it reads the associated rows > and post-filters them to incorporate partial writes, tombstones, etc. > However, those reads are not currently updating the local table latency > metrics. It should be simple enough to attach a metrics recording > transformation to the iterator produced by querying local storage. (I've > attached a patch that should apply cleanly to trunk, but there may be a > better way...) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19367) Refactor SAI so the selection of the index type is not scattered to multiple places
[
https://issues.apache.org/jira/browse/CASSANDRA-19367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17814485#comment-17814485
]
Mike Adamson commented on CASSANDRA-19367:
--
The obvious place for this would be in {{IndexTermType}} where we could replace
{{isLiteral}} and {{isVector}} with a {{getStrategy}} (or some such). The
{{Strategy}} would then need to handle all the conditionals where the above
methods are used.
Apart from anything this would tidy a lot of the current code paths where we
are constantly checking for the index type.
> Refactor SAI so the selection of the index type is not scattered to multiple
> places
> ---
>
> Key: CASSANDRA-19367
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19367
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/2i Index
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 5.x
>
>
> For Accord we want to write an internal index and finding plugging into SAI
> is a bit more channeling than it could be… we need to find multiple places
> where the SAI code “infer” the index type so it can delegate… this logic
> should be done once and made pluggable so custom SAI indexes can be defined
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19084) Test Failure: IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
[
https://issues.apache.org/jira/browse/CASSANDRA-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799835#comment-17799835
]
Mike Adamson commented on CASSANDRA-19084:
--
Committed as:
https://github.com/apache/cassandra/commit/7447ee5bddb31ea71a232a44d64dbb7dd0010708
> Test Failure:
> IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
> ---
>
> Key: CASSANDRA-19084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19084
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Michael Semb Wever
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta2, 5.0-rc, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Flakies
> https://app.circleci.com/pipelines/github/adelapena/cassandra/3329/workflows/f2124edd-fa0e-4bc5-ab03-ddfb886bf015/jobs/93097/tests
> {noformat}
> java.lang.NullPointerException
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
> at java.base/java.net.URL.openStream(URL.java:1165)
> at
> java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4009)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:154)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegateForStartup(AbstractCluster.java:292)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:410)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:383)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterStreaming(IndexStreamingFailureTest.java:123)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedNonEntireFileStreaming(IndexStreamingFailureTest.java:79)
> {noformat}
> {noformat}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:285)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedEntireFileStreaming(IndexStreamingFailureTest.java:85)
> {noformat}
> https://ci-cassandra.apache.org/job/Cassandra-5.0/106/testReport/org.apache.cassandra.distributed.test.sai/IndexStreamingFailureTest/testAvailabilityAfterFailedNonEntireFileStreaming__jdk11_x86_64_novnode/
> {noformat}
> java.lang.RuntimeException: The class file could not be written
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4021)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:155)
> at
>
[jira] [Updated] (CASSANDRA-19084) Test Failure: IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
[
https://issues.apache.org/jira/browse/CASSANDRA-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-19084:
-
Fix Version/s: 5.0-beta2
Since Version: 5.0-alpha1
Source Control Link: https://github.com/apache/cassandra/pull/3007
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Test Failure:
> IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
> ---
>
> Key: CASSANDRA-19084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19084
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Michael Semb Wever
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta2, 5.0-rc, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Flakies
> https://app.circleci.com/pipelines/github/adelapena/cassandra/3329/workflows/f2124edd-fa0e-4bc5-ab03-ddfb886bf015/jobs/93097/tests
> {noformat}
> java.lang.NullPointerException
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
> at java.base/java.net.URL.openStream(URL.java:1165)
> at
> java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4009)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:154)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegateForStartup(AbstractCluster.java:292)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:410)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:383)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterStreaming(IndexStreamingFailureTest.java:123)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedNonEntireFileStreaming(IndexStreamingFailureTest.java:79)
> {noformat}
> {noformat}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:285)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedEntireFileStreaming(IndexStreamingFailureTest.java:85)
> {noformat}
> https://ci-cassandra.apache.org/job/Cassandra-5.0/106/testReport/org.apache.cassandra.distributed.test.sai/IndexStreamingFailureTest/testAvailabilityAfterFailedNonEntireFileStreaming__jdk11_x86_64_novnode/
> {noformat}
> java.lang.RuntimeException: The class file could not be written
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4021)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
>
[jira] [Updated] (CASSANDRA-19084) Test Failure: IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
[
https://issues.apache.org/jira/browse/CASSANDRA-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-19084:
-
Status: Ready to Commit (was: Review In Progress)
> Test Failure:
> IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
> ---
>
> Key: CASSANDRA-19084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19084
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Michael Semb Wever
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Flakies
> https://app.circleci.com/pipelines/github/adelapena/cassandra/3329/workflows/f2124edd-fa0e-4bc5-ab03-ddfb886bf015/jobs/93097/tests
> {noformat}
> java.lang.NullPointerException
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
> at java.base/java.net.URL.openStream(URL.java:1165)
> at
> java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4009)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:154)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegateForStartup(AbstractCluster.java:292)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:410)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:383)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterStreaming(IndexStreamingFailureTest.java:123)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedNonEntireFileStreaming(IndexStreamingFailureTest.java:79)
> {noformat}
> {noformat}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:285)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedEntireFileStreaming(IndexStreamingFailureTest.java:85)
> {noformat}
> https://ci-cassandra.apache.org/job/Cassandra-5.0/106/testReport/org.apache.cassandra.distributed.test.sai/IndexStreamingFailureTest/testAvailabilityAfterFailedNonEntireFileStreaming__jdk11_x86_64_novnode/
> {noformat}
> java.lang.RuntimeException: The class file could not be written
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4021)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:155)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
>
[jira] [Updated] (CASSANDRA-19240) Multiple on-disk index versions on a single SSTable are not handled correctly
[ https://issues.apache.org/jira/browse/CASSANDRA-19240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19240: - Description: The SAI versioning does not consider having multiple on-disk per-column index versions against a single SSTable. The IndexDescriptor handles the versioning of all the index components for an SSTable but does not recognise multiple on-disk per-column versions. The IndexDescriptor needs to look at all on-disk components and maintain a mapping of component to version. NOTE: This is a precursor ticket to any ticket involving an on-disk version change. was: The SAI versioning does not consider having multiple on-disk per-column index versions against a single SSTable. The IndexDescriptor handles the versioning of all the index components for an SSTable but does not recognise multiple on-disk per-column versions. The IndexDescriptor needs to look at all on-disk components and maintain a mapping of component to version. > Multiple on-disk index versions on a single SSTable are not handled correctly > - > > Key: CASSANDRA-19240 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19240 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Priority: Normal > > The SAI versioning does not consider having multiple on-disk per-column index > versions against a single SSTable. > The IndexDescriptor handles the versioning of all the index components for an > SSTable but does not recognise multiple on-disk per-column versions. > The IndexDescriptor needs to look at all on-disk components and maintain a > mapping of component to version. > NOTE: This is a precursor ticket to any ticket involving an on-disk version > change. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19240) Multiple on-disk index versions on a single SSTable are not handled correctly
Mike Adamson created CASSANDRA-19240: Summary: Multiple on-disk index versions on a single SSTable are not handled correctly Key: CASSANDRA-19240 URL: https://issues.apache.org/jira/browse/CASSANDRA-19240 Project: Cassandra Issue Type: Bug Components: Feature/SAI Reporter: Mike Adamson The SAI versioning does not consider having multiple on-disk per-column index versions against a single SSTable. The IndexDescriptor handles the versioning of all the index components for an SSTable but does not recognise multiple on-disk per-column versions. The IndexDescriptor needs to look at all on-disk components and maintain a mapping of component to version. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18492) Allow full indexing on frozen collections
[ https://issues.apache.org/jira/browse/CASSANDRA-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18492: - Epic Link: CASSANDRA-19224 (was: CASSANDRA-18473) > Allow full indexing on frozen collections > - > > Key: CASSANDRA-18492 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18492 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Mike Adamson >Priority: Normal > > SAI currently indexes frozen collections as blobs of the whole collection. It > would be nice to index frozen collections such that they can be searched on > elements in the collection like non-frozen collections. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18557) CEP-30 ANN Vector Search with SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18557: - Fix Version/s: 5.0 5.0-alpha2 > CEP-30 ANN Vector Search with SAI > - > > Key: CASSANDRA-18557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18557 > Project: Cassandra > Issue Type: Epic > Components: Feature/Vector Search >Reporter: Jonathan Ellis >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0 > > > [CEP-30|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-30%3A+Approximate+Nearest+Neighbor%28ANN%29+Vector+Search+via+Storage-Attached+Indexes] > # Implement approximate nearest neighbor (ANN) vector search capability in > Apache Cassandra using storage-attached indexes (SAI). > # Support a vector of float32 embeddings as a new CQL type. > # Add ANN search to work with normal Cassandra data flow (insertion, > updating, and deleting rows). The implementation should support adding a new > vector in log(N) time, and ANN queries in M log(N) time where N is the number > of vectors and M is the number of sstables. > # Compose with other SAI predicates. > # Scatter/gather across replicas, combining topK from each to get global > topK. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18557) CEP-30 ANN Vector Search with SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18557: - Resolution: Fixed Status: Resolved (was: Open) > CEP-30 ANN Vector Search with SAI > - > > Key: CASSANDRA-18557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18557 > Project: Cassandra > Issue Type: Epic > Components: Feature/Vector Search >Reporter: Jonathan Ellis >Assignee: Mike Adamson >Priority: Normal > > [CEP-30|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-30%3A+Approximate+Nearest+Neighbor%28ANN%29+Vector+Search+via+Storage-Attached+Indexes] > # Implement approximate nearest neighbor (ANN) vector search capability in > Apache Cassandra using storage-attached indexes (SAI). > # Support a vector of float32 embeddings as a new CQL type. > # Add ANN search to work with normal Cassandra data flow (insertion, > updating, and deleting rows). The implementation should support adding a new > vector in log(N) time, and ANN queries in M log(N) time where N is the number > of vectors and M is the number of sstables. > # Compose with other SAI predicates. > # Scatter/gather across replicas, combining topK from each to get global > topK. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19084) Test Failure: IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
[
https://issues.apache.org/jira/browse/CASSANDRA-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-19084:
-
Test and Documentation Plan: There are a set of 4 dtest repeat runs and the
full CI runs in the comments.
Status: Patch Available (was: In Progress)
> Test Failure:
> IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
> ---
>
> Key: CASSANDRA-19084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19084
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Michael Semb Wever
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Flakies
> https://app.circleci.com/pipelines/github/adelapena/cassandra/3329/workflows/f2124edd-fa0e-4bc5-ab03-ddfb886bf015/jobs/93097/tests
> {noformat}
> java.lang.NullPointerException
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
> at java.base/java.net.URL.openStream(URL.java:1165)
> at
> java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4009)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:154)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegateForStartup(AbstractCluster.java:292)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:410)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:383)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterStreaming(IndexStreamingFailureTest.java:123)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedNonEntireFileStreaming(IndexStreamingFailureTest.java:79)
> {noformat}
> {noformat}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:285)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedEntireFileStreaming(IndexStreamingFailureTest.java:85)
> {noformat}
> https://ci-cassandra.apache.org/job/Cassandra-5.0/106/testReport/org.apache.cassandra.distributed.test.sai/IndexStreamingFailureTest/testAvailabilityAfterFailedNonEntireFileStreaming__jdk11_x86_64_novnode/
> {noformat}
> java.lang.RuntimeException: The class file could not be written
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4021)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:155)
> at
>
[jira] [Commented] (CASSANDRA-19084) Test Failure: IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
[
https://issues.apache.org/jira/browse/CASSANDRA-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799508#comment-17799508
]
Mike Adamson commented on CASSANDRA-19084:
--
Patch and CI runs:
|[5.0|https://github.com/apache/cassandra/pull/3007]|[j11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/449/workflows/63921ac6-7ce6-4852-b62e-38dedf0adca0]|[j17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/449/workflows/07124d06-329c-4c0f-866e-d54caa2fd47e]|
|[trunk|https://github.com/apache/cassandra/pull/3009]|[j11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/450/workflows/7c609813-4180-4a38-bf00-1bf9a4e18a6f]|[j17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/450/workflows/3adff1ad-d849-4ac0-9523-4f26302b119d]|
> Test Failure:
> IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
> ---
>
> Key: CASSANDRA-19084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19084
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Michael Semb Wever
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Flakies
> https://app.circleci.com/pipelines/github/adelapena/cassandra/3329/workflows/f2124edd-fa0e-4bc5-ab03-ddfb886bf015/jobs/93097/tests
> {noformat}
> java.lang.NullPointerException
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
> at java.base/java.net.URL.openStream(URL.java:1165)
> at
> java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4009)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:154)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegateForStartup(AbstractCluster.java:292)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:410)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:383)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterStreaming(IndexStreamingFailureTest.java:123)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedNonEntireFileStreaming(IndexStreamingFailureTest.java:79)
> {noformat}
> {noformat}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:285)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedEntireFileStreaming(IndexStreamingFailureTest.java:85)
> {noformat}
> https://ci-cassandra.apache.org/job/Cassandra-5.0/106/testReport/org.apache.cassandra.distributed.test.sai/IndexStreamingFailureTest/testAvailabilityAfterFailedNonEntireFileStreaming__jdk11_x86_64_novnode/
> {noformat}
> java.lang.RuntimeException: The class file could not be written
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4021)
> at
>
[jira] [Comment Edited] (CASSANDRA-19084) Test Failure: IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
[
https://issues.apache.org/jira/browse/CASSANDRA-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799404#comment-17799404
]
Mike Adamson edited comment on CASSANDRA-19084 at 12/21/23 5:02 PM:
Run 1:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/449/workflows/63921ac6-7ce6-4852-b62e-38dedf0adca0/jobs/41775
Run 2:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/449/workflows/f585d0a3-2f4d-44f7-a188-ef4e739f5e10/jobs/41798
Run 3:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/449/workflows/d21959f0-f1c2-4c74-99cf-a8f19fa49025/jobs/41799
Run 4:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/449/workflows/e6fad111-87f4-4ef7-b2f6-70b19936518a/jobs/41800
was (Author: mike_tr_adamson):
Run 1:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/449/workflows/63921ac6-7ce6-4852-b62e-38dedf0adca0/jobs/41775
Run 2:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/449/workflows/f585d0a3-2f4d-44f7-a188-ef4e739f5e10/jobs/41798
> Test Failure:
> IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
> ---
>
> Key: CASSANDRA-19084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19084
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Michael Semb Wever
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Flakies
> https://app.circleci.com/pipelines/github/adelapena/cassandra/3329/workflows/f2124edd-fa0e-4bc5-ab03-ddfb886bf015/jobs/93097/tests
> {noformat}
> java.lang.NullPointerException
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
> at java.base/java.net.URL.openStream(URL.java:1165)
> at
> java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4009)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:154)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegateForStartup(AbstractCluster.java:292)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:410)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:383)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterStreaming(IndexStreamingFailureTest.java:123)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedNonEntireFileStreaming(IndexStreamingFailureTest.java:79)
> {noformat}
> {noformat}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:285)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedEntireFileStreaming(IndexStreamingFailureTest.java:85)
> {noformat}
>
[jira] [Comment Edited] (CASSANDRA-19084) Test Failure: IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
[
https://issues.apache.org/jira/browse/CASSANDRA-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799404#comment-17799404
]
Mike Adamson edited comment on CASSANDRA-19084 at 12/21/23 4:49 PM:
Run 1:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/449/workflows/63921ac6-7ce6-4852-b62e-38dedf0adca0/jobs/41775
Run 2:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/449/workflows/f585d0a3-2f4d-44f7-a188-ef4e739f5e10/jobs/41798
was (Author: mike_tr_adamson):
Run 1:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/449/workflows/63921ac6-7ce6-4852-b62e-38dedf0adca0/jobs/41775
> Test Failure:
> IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
> ---
>
> Key: CASSANDRA-19084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19084
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Michael Semb Wever
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Flakies
> https://app.circleci.com/pipelines/github/adelapena/cassandra/3329/workflows/f2124edd-fa0e-4bc5-ab03-ddfb886bf015/jobs/93097/tests
> {noformat}
> java.lang.NullPointerException
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
> at java.base/java.net.URL.openStream(URL.java:1165)
> at
> java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4009)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:154)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegateForStartup(AbstractCluster.java:292)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:410)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:383)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterStreaming(IndexStreamingFailureTest.java:123)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedNonEntireFileStreaming(IndexStreamingFailureTest.java:79)
> {noformat}
> {noformat}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:285)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedEntireFileStreaming(IndexStreamingFailureTest.java:85)
> {noformat}
> https://ci-cassandra.apache.org/job/Cassandra-5.0/106/testReport/org.apache.cassandra.distributed.test.sai/IndexStreamingFailureTest/testAvailabilityAfterFailedNonEntireFileStreaming__jdk11_x86_64_novnode/
> {noformat}
> java.lang.RuntimeException: The class file could not be written
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4021)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
>
[jira] [Comment Edited] (CASSANDRA-19084) Test Failure: IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
[
https://issues.apache.org/jira/browse/CASSANDRA-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799404#comment-17799404
]
Mike Adamson edited comment on CASSANDRA-19084 at 12/21/23 3:02 PM:
Run 1:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/449/workflows/63921ac6-7ce6-4852-b62e-38dedf0adca0/jobs/41775
was (Author: mike_tr_adamson):
Run 1:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/446/workflows/6441919f-0de0-4d5d-8a44-407535d3605e
> Test Failure:
> IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
> ---
>
> Key: CASSANDRA-19084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19084
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Michael Semb Wever
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Flakies
> https://app.circleci.com/pipelines/github/adelapena/cassandra/3329/workflows/f2124edd-fa0e-4bc5-ab03-ddfb886bf015/jobs/93097/tests
> {noformat}
> java.lang.NullPointerException
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
> at java.base/java.net.URL.openStream(URL.java:1165)
> at
> java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4009)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:154)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegateForStartup(AbstractCluster.java:292)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:410)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:383)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterStreaming(IndexStreamingFailureTest.java:123)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedNonEntireFileStreaming(IndexStreamingFailureTest.java:79)
> {noformat}
> {noformat}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:285)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedEntireFileStreaming(IndexStreamingFailureTest.java:85)
> {noformat}
> https://ci-cassandra.apache.org/job/Cassandra-5.0/106/testReport/org.apache.cassandra.distributed.test.sai/IndexStreamingFailureTest/testAvailabilityAfterFailedNonEntireFileStreaming__jdk11_x86_64_novnode/
> {noformat}
> java.lang.RuntimeException: The class file could not be written
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4021)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
>
[jira] [Commented] (CASSANDRA-19084) Test Failure: IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
[
https://issues.apache.org/jira/browse/CASSANDRA-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799402#comment-17799402
]
Mike Adamson commented on CASSANDRA-19084:
--
It looks like all the failures I can find associated with this issue result
from a node restart in the test, where the ByteBuddy injection fails for
several different reasons. There are no failures on cluster startup. It should
be pointed out here that the injection is expected to be disabled after the
restart so is not really needed.
To try and fix this, I have restructured the inject logic to only inject on the
first start of the node and not inject on any further starts. This looks to
have fixed the issue but I am repeating the test runs a number of times to make
sure of this.
[~adelapena] I'd like you to have a look at this to make sure that my
assumption about the injection not being needed is correct at all points after
the restart.
> Test Failure:
> IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
> ---
>
> Key: CASSANDRA-19084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19084
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Michael Semb Wever
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> Flakies
> https://app.circleci.com/pipelines/github/adelapena/cassandra/3329/workflows/f2124edd-fa0e-4bc5-ab03-ddfb886bf015/jobs/93097/tests
> {noformat}
> java.lang.NullPointerException
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
> at java.base/java.net.URL.openStream(URL.java:1165)
> at
> java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4009)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:154)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegateForStartup(AbstractCluster.java:292)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:410)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:383)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterStreaming(IndexStreamingFailureTest.java:123)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedNonEntireFileStreaming(IndexStreamingFailureTest.java:79)
> {noformat}
> {noformat}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:285)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedEntireFileStreaming(IndexStreamingFailureTest.java:85)
> {noformat}
> https://ci-cassandra.apache.org/job/Cassandra-5.0/106/testReport/org.apache.cassandra.distributed.test.sai/IndexStreamingFailureTest/testAvailabilityAfterFailedNonEntireFileStreaming__jdk11_x86_64_novnode/
> {noformat}
> java.lang.RuntimeException: The class file could not be written
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4021)
>
[jira] [Commented] (CASSANDRA-19084) Test Failure: IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
[
https://issues.apache.org/jira/browse/CASSANDRA-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799404#comment-17799404
]
Mike Adamson commented on CASSANDRA-19084:
--
Run 1:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/446/workflows/6441919f-0de0-4d5d-8a44-407535d3605e
> Test Failure:
> IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
> ---
>
> Key: CASSANDRA-19084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19084
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Michael Semb Wever
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> Flakies
> https://app.circleci.com/pipelines/github/adelapena/cassandra/3329/workflows/f2124edd-fa0e-4bc5-ab03-ddfb886bf015/jobs/93097/tests
> {noformat}
> java.lang.NullPointerException
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
> at java.base/java.net.URL.openStream(URL.java:1165)
> at
> java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4009)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:154)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegateForStartup(AbstractCluster.java:292)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:410)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:383)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterStreaming(IndexStreamingFailureTest.java:123)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedNonEntireFileStreaming(IndexStreamingFailureTest.java:79)
> {noformat}
> {noformat}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:285)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedEntireFileStreaming(IndexStreamingFailureTest.java:85)
> {noformat}
> https://ci-cassandra.apache.org/job/Cassandra-5.0/106/testReport/org.apache.cassandra.distributed.test.sai/IndexStreamingFailureTest/testAvailabilityAfterFailedNonEntireFileStreaming__jdk11_x86_64_novnode/
> {noformat}
> java.lang.RuntimeException: The class file could not be written
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4021)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:155)
> at
>
[jira] [Assigned] (CASSANDRA-19084) Test Failure: IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
[
https://issues.apache.org/jira/browse/CASSANDRA-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson reassigned CASSANDRA-19084:
Assignee: Mike Adamson
> Test Failure:
> IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
> ---
>
> Key: CASSANDRA-19084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19084
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Michael Semb Wever
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> Flakies
> https://app.circleci.com/pipelines/github/adelapena/cassandra/3329/workflows/f2124edd-fa0e-4bc5-ab03-ddfb886bf015/jobs/93097/tests
> {noformat}
> java.lang.NullPointerException
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
> at java.base/java.net.URL.openStream(URL.java:1165)
> at
> java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4009)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:154)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegateForStartup(AbstractCluster.java:292)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:410)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:383)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterStreaming(IndexStreamingFailureTest.java:123)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedNonEntireFileStreaming(IndexStreamingFailureTest.java:79)
> {noformat}
> {noformat}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:285)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedEntireFileStreaming(IndexStreamingFailureTest.java:85)
> {noformat}
> https://ci-cassandra.apache.org/job/Cassandra-5.0/106/testReport/org.apache.cassandra.distributed.test.sai/IndexStreamingFailureTest/testAvailabilityAfterFailedNonEntireFileStreaming__jdk11_x86_64_novnode/
> {noformat}
> java.lang.RuntimeException: The class file could not be written
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4021)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:155)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
>
[jira] [Updated] (CASSANDRA-18598) Use trie for literal index building
[ https://issues.apache.org/jira/browse/CASSANDRA-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18598: - Fix Version/s: 5.0-beta2 (was: 5.x) Source Control Link: https://github.com/apache/cassandra/pull/2989 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as https://github.com/apache/cassandra/commit/3b05051f8678c28bc9d93a89123c68f8d0b93b7b > Use trie for literal index building > --- > > Key: CASSANDRA-18598 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18598 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta2 > > Time Spent: 7h 50m > Remaining Estimate: 0h > > The BlockBalancedTreeRamBuffer for numerics indexes uses a trie for building > indexes in memory before flushing to disk. The RAMStringIndexer for literal > indexes does not do this. It uses lucene classes to build the in memory index > before flushing. > We should combine both of the in-memory index builders and have them both use > the trie. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18598) Use trie for literal index building
[ https://issues.apache.org/jira/browse/CASSANDRA-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18598: - Status: Ready to Commit (was: Review In Progress) Final CI runs: |[5.0|https://github.com/apache/cassandra/pull/2989]|[j11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/443/workflows/f5ab3823-ac34-41df-aefb-7779a7359735]|[j17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/443/workflows/f64736ff-8f79-4b98-bee4-6831188cb720]| |[trunk|https://github.com/apache/cassandra/pull/2991]|[j11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/444/workflows/6c53718c-1898-449b-b2f3-4c1f365fd057]|[j17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/444/workflows/a19ee49b-76bd-452c-8ccd-4bdc68341264]| > Use trie for literal index building > --- > > Key: CASSANDRA-18598 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18598 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.x > > Time Spent: 7h 50m > Remaining Estimate: 0h > > The BlockBalancedTreeRamBuffer for numerics indexes uses a trie for building > indexes in memory before flushing to disk. The RAMStringIndexer for literal > indexes does not do this. It uses lucene classes to build the in memory index > before flushing. > We should combine both of the in-memory index builders and have them both use > the trie. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18165) Investigate removing PriorityQueue usage from KeyRangeConcatIterator
[
https://issues.apache.org/jira/browse/CASSANDRA-18165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-18165:
-
Change Category: Code Clarity
Complexity: Low Hanging Fruit
Fix Version/s: 5.x
Status: Open (was: Triage Needed)
> Investigate removing PriorityQueue usage from KeyRangeConcatIterator
>
>
> Key: CASSANDRA-18165
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18165
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/SAI
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Normal
> Labels: SAI
> Fix For: 5.x
>
>
> It has been identified during the review of CASSANDRA-18058 that the
> KeyRangeConcatIterator could potentially stop using a PriorityQueue to
> maintain it's active list of sorted KeyRangeIterators.
> The code suggested by [~maedhroz] is as follows:
> {code:java}
> private int i = 0;
> ...
> protected void performSkipTo(PrimaryKey primaryKey)
> {
> while (i < toRelease.size())
> {
> RangeIterator currentIterator = toRelease.get(i);
> if (currentIterator.getCurrent().compareTo(primaryKey) >= 0)
> break;
> if (currentIterator.getMaximum().compareTo(primaryKey) >= 0)
> {
> currentIterator.skipTo(primaryKey);
> break;
> }
> i++;
> }
> }
> ...
> protected PrimaryKey computeNext()
> {
> while (i < toRelease.size())
> {
> RangeIterator currentIterator = toRelease.get(i);
>
> if (currentIterator.hasNext())
> return currentIterator.next();
>
> i++;
> }
> return endOfData();
> }
> {code}
> It was decided that this change would need performance and correctness
> testing in it's own right would not be included in the original SAI CEP
> ticket.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18165) Investigate removing PriorityQueue usage from KeyRangeConcatIterator
[
https://issues.apache.org/jira/browse/CASSANDRA-18165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson reassigned CASSANDRA-18165:
Assignee: Mike Adamson
> Investigate removing PriorityQueue usage from KeyRangeConcatIterator
>
>
> Key: CASSANDRA-18165
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18165
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/SAI
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Normal
> Labels: SAI
>
> It has been identified during the review of CASSANDRA-18058 that the
> KeyRangeConcatIterator could potentially stop using a PriorityQueue to
> maintain it's active list of sorted KeyRangeIterators.
> The code suggested by [~maedhroz] is as follows:
> {code:java}
> private int i = 0;
> ...
> protected void performSkipTo(PrimaryKey primaryKey)
> {
> while (i < toRelease.size())
> {
> RangeIterator currentIterator = toRelease.get(i);
> if (currentIterator.getCurrent().compareTo(primaryKey) >= 0)
> break;
> if (currentIterator.getMaximum().compareTo(primaryKey) >= 0)
> {
> currentIterator.skipTo(primaryKey);
> break;
> }
> i++;
> }
> }
> ...
> protected PrimaryKey computeNext()
> {
> while (i < toRelease.size())
> {
> RangeIterator currentIterator = toRelease.get(i);
>
> if (currentIterator.hasNext())
> return currentIterator.next();
>
> i++;
> }
> return endOfData();
> }
> {code}
> It was decided that this change would need performance and correctness
> testing in it's own right would not be included in the original SAI CEP
> ticket.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18598) Use trie for literal index building
[ https://issues.apache.org/jira/browse/CASSANDRA-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18598: - Test and Documentation Plan: CI runs are in the comments Status: Patch Available (was: In Progress) > Use trie for literal index building > --- > > Key: CASSANDRA-18598 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18598 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.x > > Time Spent: 0.5h > Remaining Estimate: 0h > > The BlockBalancedTreeRamBuffer for numerics indexes uses a trie for building > indexes in memory before flushing to disk. The RAMStringIndexer for literal > indexes does not do this. It uses lucene classes to build the in memory index > before flushing. > We should combine both of the in-memory index builders and have them both use > the trie. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19177) SAI query timeouts can cause resource leaks
[ https://issues.apache.org/jira/browse/CASSANDRA-19177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19177: - Fix Version/s: 5.0-beta2 Since Version: 5.0-alpha1 Source Control Link: https://github.com/apache/cassandra/pull/2986 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as https://github.com/apache/cassandra/commit/5ec0669bec0a1cda46b0079a6043bd6ef12b3056 > SAI query timeouts can cause resource leaks > --- > > Key: CASSANDRA-19177 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19177 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta2 > > Time Spent: 1h > Remaining Estimate: 0h > > There are several places in the SAI query path where a query timeout can > result in a resource not being closed correctly. We need to make sure that > wherever QueryContext.checkpoint is called we catch the resulting exception > and close any open resources. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19177) SAI query timeouts can cause resource leaks
[ https://issues.apache.org/jira/browse/CASSANDRA-19177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19177: - Reviewers: Caleb Rackliffe, Mike Adamson (was: Caleb Rackliffe) Status: Review In Progress (was: Patch Available) > SAI query timeouts can cause resource leaks > --- > > Key: CASSANDRA-19177 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19177 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Time Spent: 1h > Remaining Estimate: 0h > > There are several places in the SAI query path where a query timeout can > result in a resource not being closed correctly. We need to make sure that > wherever QueryContext.checkpoint is called we catch the resulting exception > and close any open resources. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19177) SAI query timeouts can cause resource leaks
[ https://issues.apache.org/jira/browse/CASSANDRA-19177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19177: - Status: Ready to Commit (was: Review In Progress) > SAI query timeouts can cause resource leaks > --- > > Key: CASSANDRA-19177 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19177 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Time Spent: 1h > Remaining Estimate: 0h > > There are several places in the SAI query path where a query timeout can > result in a resource not being closed correctly. We need to make sure that > wherever QueryContext.checkpoint is called we catch the resulting exception > and close any open resources. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19177) SAI query timeouts can cause resource leaks
[ https://issues.apache.org/jira/browse/CASSANDRA-19177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19177: - Status: Needs Committer (was: Review In Progress) > SAI query timeouts can cause resource leaks > --- > > Key: CASSANDRA-19177 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19177 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Time Spent: 1h > Remaining Estimate: 0h > > There are several places in the SAI query path where a query timeout can > result in a resource not being closed correctly. We need to make sure that > wherever QueryContext.checkpoint is called we catch the resulting exception > and close any open resources. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19177) SAI query timeouts can cause resource leaks
[ https://issues.apache.org/jira/browse/CASSANDRA-19177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19177: - Status: Patch Available (was: Needs Committer) > SAI query timeouts can cause resource leaks > --- > > Key: CASSANDRA-19177 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19177 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Time Spent: 1h > Remaining Estimate: 0h > > There are several places in the SAI query path where a query timeout can > result in a resource not being closed correctly. We need to make sure that > wherever QueryContext.checkpoint is called we catch the resulting exception > and close any open resources. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18598) Use trie for literal index building
[ https://issues.apache.org/jira/browse/CASSANDRA-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18598: - Change Category: Operability Complexity: Normal Component/s: Feature/SAI Status: Open (was: Triage Needed) > Use trie for literal index building > --- > > Key: CASSANDRA-18598 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18598 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.x > > > The BlockBalancedTreeRamBuffer for numerics indexes uses a trie for building > indexes in memory before flushing to disk. The RAMStringIndexer for literal > indexes does not do this. It uses lucene classes to build the in memory index > before flushing. > We should combine both of the in-memory index builders and have them both use > the trie. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19177) SAI query timeouts can cause resource leaks
[ https://issues.apache.org/jira/browse/CASSANDRA-19177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19177: - Test and Documentation Plan: CI runs are in the comments Status: Patch Available (was: In Progress) > SAI query timeouts can cause resource leaks > --- > > Key: CASSANDRA-19177 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19177 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Time Spent: 40m > Remaining Estimate: 0h > > There are several places in the SAI query path where a query timeout can > result in a resource not being closed correctly. We need to make sure that > wherever QueryContext.checkpoint is called we catch the resulting exception > and close any open resources. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19177) SAI query timeouts can cause resource leaks
[ https://issues.apache.org/jira/browse/CASSANDRA-19177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795775#comment-17795775 ] Mike Adamson commented on CASSANDRA-19177: -- The patches for this are here: |[5.0|https://github.com/apache/cassandra/pull/2986|]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/414/workflows/937554f1-78d3-4786-9d75-3893610ca475]| |[trunk|https://github.com/apache/cassandra/pull/2987]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/415/workflows/2d50ac59-3383-4066-aabc-4542a2425d13]| > SAI query timeouts can cause resource leaks > --- > > Key: CASSANDRA-19177 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19177 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Time Spent: 40m > Remaining Estimate: 0h > > There are several places in the SAI query path where a query timeout can > result in a resource not being closed correctly. We need to make sure that > wherever QueryContext.checkpoint is called we catch the resulting exception > and close any open resources. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18940) SAI post-filtering reads don't update local table latency metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795692#comment-17795692 ] Mike Adamson commented on CASSANDRA-18940: -- I am stopping working on this for the time being. Having looked at the existing metrics we already have a count of the local reads with TotalPartitionReads. This metric is incremented for each call to SinglePartitionReadCommand.queryMemtableAndDisk. Note that TotalPartitionReads is distinct from TotalRowReads because we still do full partition scans for static column indexes. SAI doesn't have any latency metrics, so I would prefer that we either repurpose this ticket or create a new one to look at adding latency metrics to SAI. > SAI post-filtering reads don't update local table latency metrics > - > > Key: CASSANDRA-18940 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18940 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index, Feature/SAI, Observability/Metrics >Reporter: Caleb Rackliffe >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-rc, 5.x > > Attachments: > draft_fix_for_SAI_post-filtering_reads_not_updating_local_table_metrics.patch > > > Once an SAI index finds matches (primary keys), it reads the associated rows > and post-filters them to incorporate partial writes, tombstones, etc. > However, those reads are not currently updating the local table latency > metrics. It should be simple enough to attach a metrics recording > transformation to the iterator produced by querying local storage. (I've > attached a patch that should apply cleanly to trunk, but there may be a > better way...) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19095) Test Failure: org.apache.cassandra.index.sai.cql.VectorSegmentationTest.testMultipleSegmentsForCompaction
[
https://issues.apache.org/jira/browse/CASSANDRA-19095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794809#comment-17794809
]
Mike Adamson commented on CASSANDRA-19095:
--
I have done a quick investigation of CQLVectorTest.sandwichBetweenUDTs locally.
I can get the failure to reproduce if the whole class is run repeatedly but I
can get it to repeat if the test is run on its own. The failure is the same
each time and reflects the failures seen in CI
{code:java}
org.apache.cassandra.schema.SchemaKeyspace$MissingColumns: Columns not found in
schema table for cql_test_keyspace.table_20
at
org.apache.cassandra.schema.SchemaKeyspace.fetchColumns(SchemaKeyspace.java:1081)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:1032)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:991)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:951)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:941)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:929)
at
org.apache.cassandra.cql3.validation.operations.CQLVectorTest.sandwichBetweenUDTs(CQLVectorTest.java:199)
{code}
It is always table_20 that is the failure point.
I'm just marking up the rest of the test to find out which one creates table_20
and I'll report back.
> Test Failure:
> org.apache.cassandra.index.sai.cql.VectorSegmentationTest.testMultipleSegmentsForCompaction
> -
>
> Key: CASSANDRA-19095
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19095
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Michael Semb Wever
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> Seen in j11_utests_compression in CASSANDRA-19034
> https://app.circleci.com/pipelines/github/michaelsembwever/cassandra/259/workflows/f343d3e3-00cf-4e13-bb4d-bbfff1d3658c/jobs/21135/tests
> {noformat}
> junit.framework.AssertionFailedError:
> Expecting actual:
> 0.9667
> to be greater than or equal to:
> 0.99
> at
> org.apache.cassandra.index.sai.cql.VectorSegmentationTest.testMultipleSegmentsForCompaction(VectorSegmentationTest.java:107)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19186) Investigate p99 query spikes during memtable index flushes
[ https://issues.apache.org/jira/browse/CASSANDRA-19186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19186: - Fix Version/s: 5.x > Investigate p99 query spikes during memtable index flushes > -- > > Key: CASSANDRA-19186 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19186 > Project: Cassandra > Issue Type: Task > Components: Feature/SAI >Reporter: Mike Adamson >Priority: Normal > Fix For: 5.x > > Attachments: Screenshot from 2023-12-08 14-55-12.png > > > The attached image shows a clear p99 spike when a memtable index is being > flushed to disk. This latency spike should be investigated to see if it can > be mitigated. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19186) Investigate p99 query spikes during memtable index flushes
Mike Adamson created CASSANDRA-19186: Summary: Investigate p99 query spikes during memtable index flushes Key: CASSANDRA-19186 URL: https://issues.apache.org/jira/browse/CASSANDRA-19186 Project: Cassandra Issue Type: Task Components: Feature/SAI Reporter: Mike Adamson Attachments: Screenshot from 2023-12-08 14-55-12.png The attached image shows a clear p99 spike when a memtable index is being flushed to disk. This latency spike should be investigated to see if it can be mitigated. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19095) Test Failure: org.apache.cassandra.index.sai.cql.VectorSegmentationTest.testMultipleSegmentsForCompaction
[
https://issues.apache.org/jira/browse/CASSANDRA-19095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794701#comment-17794701
]
Mike Adamson commented on CASSANDRA-19095:
--
You are correct, it was forgotten. I have now raised CASSANDRA-19185 to
investigate these failures. I should point out that the
CQLVectorTest.sandwichBetweenUDTs was introduced by CASSANDRA-18964 and did not
form part of the CASSANDRA-18715 CEP-30 ticket. I will wait for the current CI
runs to complete before raising a ticket for this failure.
> Test Failure:
> org.apache.cassandra.index.sai.cql.VectorSegmentationTest.testMultipleSegmentsForCompaction
> -
>
> Key: CASSANDRA-19095
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19095
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Michael Semb Wever
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> Seen in j11_utests_compression in CASSANDRA-19034
> https://app.circleci.com/pipelines/github/michaelsembwever/cassandra/259/workflows/f343d3e3-00cf-4e13-bb4d-bbfff1d3658c/jobs/21135/tests
> {noformat}
> junit.framework.AssertionFailedError:
> Expecting actual:
> 0.9667
> to be greater than or equal to:
> 0.99
> at
> org.apache.cassandra.index.sai.cql.VectorSegmentationTest.testMultipleSegmentsForCompaction(VectorSegmentationTest.java:107)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19185) Vector search tests are failing on recall accuracy
Mike Adamson created CASSANDRA-19185: Summary: Vector search tests are failing on recall accuracy Key: CASSANDRA-19185 URL: https://issues.apache.org/jira/browse/CASSANDRA-19185 Project: Cassandra Issue Type: Bug Components: Feature/SAI Reporter: Mike Adamson Vector tests are failing -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19084) Test Failure: IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
[
https://issues.apache.org/jira/browse/CASSANDRA-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-19084:
-
Epic Link: CASSANDRA-18473
> Test Failure:
> IndexStreamingFailureTest.testAvailabilityAfterFailed*EntireFileStreaming
> ---
>
> Key: CASSANDRA-19084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19084
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Michael Semb Wever
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> Flakies
> https://app.circleci.com/pipelines/github/adelapena/cassandra/3329/workflows/f2124edd-fa0e-4bc5-ab03-ddfb886bf015/jobs/93097/tests
> {noformat}
> java.lang.NullPointerException
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
> at java.base/java.net.URL.openStream(URL.java:1165)
> at
> java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
> at
> net.bytebuddy.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4009)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:154)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegateForStartup(AbstractCluster.java:292)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:410)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:383)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterStreaming(IndexStreamingFailureTest.java:123)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedNonEntireFileStreaming(IndexStreamingFailureTest.java:79)
> {noformat}
> {noformat}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:285)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest.testAvailabilityAfterFailedEntireFileStreaming(IndexStreamingFailureTest.java:85)
> {noformat}
> https://ci-cassandra.apache.org/job/Cassandra-5.0/106/testReport/org.apache.cassandra.distributed.test.sai/IndexStreamingFailureTest/testAvailabilityAfterFailedNonEntireFileStreaming__jdk11_x86_64_novnode/
> {noformat}
> java.lang.RuntimeException: The class file could not be written
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4021)
> at
> net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4050)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3734)
> at
> net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:3986)
> at
> org.apache.cassandra.distributed.test.sai.IndexStreamingFailureTest$ByteBuddyHelper.installErrors(IndexStreamingFailureTest.java:155)
> at
> org.apache.cassandra.distributed.shared.AbstractBuilder$1.initialise(AbstractBuilder.java:360)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.newInstance(AbstractCluster.java:312)
> at
>
[jira] [Updated] (CASSANDRA-19177) SAI query timeouts can cause resource leaks
[ https://issues.apache.org/jira/browse/CASSANDRA-19177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19177: - Bug Category: Parent values: Degradation(12984) Complexity: Normal Discovered By: Code Inspection Severity: Normal Assignee: Mike Adamson Status: Open (was: Triage Needed) > SAI query timeouts can cause resource leaks > --- > > Key: CASSANDRA-19177 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19177 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > > There are several places in the SAI query path where a query timeout can > result in a resource not being closed correctly. We need to make sure that > wherever QueryContext.checkpoint is called we catch the resulting exception > and close any open resources. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19177) SAI query timeouts can cause resource leaks
Mike Adamson created CASSANDRA-19177: Summary: SAI query timeouts can cause resource leaks Key: CASSANDRA-19177 URL: https://issues.apache.org/jira/browse/CASSANDRA-19177 Project: Cassandra Issue Type: Bug Components: Feature/SAI Reporter: Mike Adamson There are several places in the SAI query path where a query timeout can result in a resource not being closed correctly. We need to make sure that wherever QueryContext.checkpoint is called we catch the resulting exception and close any open resources. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18940) SAI post-filtering reads don't update local table latency metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson reassigned CASSANDRA-18940: Assignee: Mike Adamson (was: Caleb Rackliffe) > SAI post-filtering reads don't update local table latency metrics > - > > Key: CASSANDRA-18940 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18940 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index, Feature/SAI, Observability/Metrics >Reporter: Caleb Rackliffe >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-rc, 5.x > > Attachments: > draft_fix_for_SAI_post-filtering_reads_not_updating_local_table_metrics.patch > > > Once an SAI index finds matches (primary keys), it reads the associated rows > and post-filters them to incorporate partial writes, tombstones, etc. > However, those reads are not currently updating the local table latency > metrics. It should be simple enough to attach a metrics recording > transformation to the iterator produced by querying local storage. (I've > attached a patch that should apply cleanly to trunk, but there may be a > better way...) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19011) Harry-found exception in during SAI query
[
https://issues.apache.org/jira/browse/CASSANDRA-19011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-19011:
-
Test and Documentation Plan: CI results are in the comments
Status: Patch Available (was: In Progress)
> Harry-found exception in during SAI query
> -
>
> Key: CASSANDRA-19011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19011
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Alex Petrov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Schema:
> {code:java}
> CREATE TABLE IF NOT EXISTS distributed_test_keyspace.tbl1 (pk1 bigint,ck1
> bigint,v1 ascii,v2 bigint, PRIMARY KEY (pk1, ck1)) WITH CLUSTERING ORDER BY
> (ck1 ASC);
> CREATE CUSTOM INDEX v1_sai_idx ON distributed_test_keyspace.tbl1 (v1) USING
> 'StorageAttachedIndex' WITH OPTIONS = {'case_sensitive': 'false',
> 'normalize': 'true', 'ascii': 'true'}; ;
> CREATE CUSTOM INDEX v2_sai_idx ON distributed_test_keyspace.tbl1 (v2) USING
> 'StorageAttachedIndex';
> {code}
> {code:java}
> java.lang.AssertionError: skipped to an item smaller than the target;
> iterator:
> org.apache.cassandra.index.sai.disk.IndexSearchResultIterator@f399f79, target
> key: PrimaryKey: { token: 8384965201802291970, partition:
> DecoratedKey(8384965201802291970, c4bc1c50f9e76a50), clustering:
> CLUSTERING:8b4b4c5991a4ea10 } , returned key: PrimaryKey: { token:
> 8384965201802291970, partition: DecoratedKey(8384965201802291970,
> c4bc1c50f9e76a50), clustering: CLUSTERING:89f1cf92658cb668 }
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:95)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:39)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.tryToComputeNext(AbstractGuavaIterator.java:122)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIterator.tryToComputeNext(KeyRangeIterator.java:129)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.hasNext(AbstractGuavaIterator.java:116)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKey(StorageAttachedIndexSearcher.java:274)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKeyInRange(StorageAttachedIndexSearcher.java:203)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextSelectedKeyInRange(StorageAttachedIndexSearcher.java:234)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextRowIterator(StorageAttachedIndexSearcher.java:188)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:169)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:111)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:91)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:338)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:201)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:186)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:346)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:2186)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2581)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> relocated.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829) {code}
>
> Unfortunately, there's no tooling for shrinking around SAI just yet, but I
> have a programmatic repro using INSERT and DELETE statements. I will do my
> best to post it asap, but thought this can already be useful for visibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail:
[jira] [Commented] (CASSANDRA-19011) Harry-found exception in during SAI query
[
https://issues.apache.org/jira/browse/CASSANDRA-19011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791197#comment-17791197
]
Mike Adamson commented on CASSANDRA-19011:
--
The PRs for this are below:
|[5.0|https://github.com/apache/cassandra/pull/2943]|[j11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/407/workflows/f74aadb4-23fd-4b85-a456-b4051a2e4031]|[j17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/407/workflows/d05612f9-0c76-4430-97e1-cbf2000d7cde]|
|[trunk|https://github.com/apache/cassandra/pull/2943]|[j11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/408/workflows/47dd4596-15e8-4b35-9d64-4a5625112a71]|[j17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/408/workflows/6eda2aba-8065-4644-9c03-1a0beaf5a7c8]|
> Harry-found exception in during SAI query
> -
>
> Key: CASSANDRA-19011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19011
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Alex Petrov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Schema:
> {code:java}
> CREATE TABLE IF NOT EXISTS distributed_test_keyspace.tbl1 (pk1 bigint,ck1
> bigint,v1 ascii,v2 bigint, PRIMARY KEY (pk1, ck1)) WITH CLUSTERING ORDER BY
> (ck1 ASC);
> CREATE CUSTOM INDEX v1_sai_idx ON distributed_test_keyspace.tbl1 (v1) USING
> 'StorageAttachedIndex' WITH OPTIONS = {'case_sensitive': 'false',
> 'normalize': 'true', 'ascii': 'true'}; ;
> CREATE CUSTOM INDEX v2_sai_idx ON distributed_test_keyspace.tbl1 (v2) USING
> 'StorageAttachedIndex';
> {code}
> {code:java}
> java.lang.AssertionError: skipped to an item smaller than the target;
> iterator:
> org.apache.cassandra.index.sai.disk.IndexSearchResultIterator@f399f79, target
> key: PrimaryKey: { token: 8384965201802291970, partition:
> DecoratedKey(8384965201802291970, c4bc1c50f9e76a50), clustering:
> CLUSTERING:8b4b4c5991a4ea10 } , returned key: PrimaryKey: { token:
> 8384965201802291970, partition: DecoratedKey(8384965201802291970,
> c4bc1c50f9e76a50), clustering: CLUSTERING:89f1cf92658cb668 }
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:95)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:39)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.tryToComputeNext(AbstractGuavaIterator.java:122)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIterator.tryToComputeNext(KeyRangeIterator.java:129)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.hasNext(AbstractGuavaIterator.java:116)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKey(StorageAttachedIndexSearcher.java:274)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKeyInRange(StorageAttachedIndexSearcher.java:203)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextSelectedKeyInRange(StorageAttachedIndexSearcher.java:234)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextRowIterator(StorageAttachedIndexSearcher.java:188)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:169)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:111)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:91)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:338)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:201)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:186)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:346)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:2186)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2581)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
>
[jira] [Commented] (CASSANDRA-19011) Harry-found exception in during SAI query
[
https://issues.apache.org/jira/browse/CASSANDRA-19011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791196#comment-17791196
]
Mike Adamson commented on CASSANDRA-19011:
--
Following on from the above comment I have found 2 further issues associated
with this. These we found using a randomized test that will be added with the
patch.
# The search being done in KeyLookup.clusteredSeekToKey is wrong and results in
the wrong row id being returned for the clustering key.
# The MonotonicBlockPackedReader.indexOf does not seem to be working correctly
so I have introduced a new on-disk component that stores the partition size per
partition. This will be faster than the previous mechanism because we can now
use .get to read the partition size rather than .indexOf which
uses a binary search to find the size. The overall impact on disk size should
be small because we are saving a value per partition.
> Harry-found exception in during SAI query
> -
>
> Key: CASSANDRA-19011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19011
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Alex Petrov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Schema:
> {code:java}
> CREATE TABLE IF NOT EXISTS distributed_test_keyspace.tbl1 (pk1 bigint,ck1
> bigint,v1 ascii,v2 bigint, PRIMARY KEY (pk1, ck1)) WITH CLUSTERING ORDER BY
> (ck1 ASC);
> CREATE CUSTOM INDEX v1_sai_idx ON distributed_test_keyspace.tbl1 (v1) USING
> 'StorageAttachedIndex' WITH OPTIONS = {'case_sensitive': 'false',
> 'normalize': 'true', 'ascii': 'true'}; ;
> CREATE CUSTOM INDEX v2_sai_idx ON distributed_test_keyspace.tbl1 (v2) USING
> 'StorageAttachedIndex';
> {code}
> {code:java}
> java.lang.AssertionError: skipped to an item smaller than the target;
> iterator:
> org.apache.cassandra.index.sai.disk.IndexSearchResultIterator@f399f79, target
> key: PrimaryKey: { token: 8384965201802291970, partition:
> DecoratedKey(8384965201802291970, c4bc1c50f9e76a50), clustering:
> CLUSTERING:8b4b4c5991a4ea10 } , returned key: PrimaryKey: { token:
> 8384965201802291970, partition: DecoratedKey(8384965201802291970,
> c4bc1c50f9e76a50), clustering: CLUSTERING:89f1cf92658cb668 }
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:95)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:39)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.tryToComputeNext(AbstractGuavaIterator.java:122)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIterator.tryToComputeNext(KeyRangeIterator.java:129)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.hasNext(AbstractGuavaIterator.java:116)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKey(StorageAttachedIndexSearcher.java:274)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKeyInRange(StorageAttachedIndexSearcher.java:203)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextSelectedKeyInRange(StorageAttachedIndexSearcher.java:234)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextRowIterator(StorageAttachedIndexSearcher.java:188)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:169)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:111)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:91)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:338)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:201)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:186)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:346)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:2186)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2581)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at
[jira] [Commented] (CASSANDRA-19011) Harry-found exception in during SAI query
[
https://issues.apache.org/jira/browse/CASSANDRA-19011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790521#comment-17790521
]
Mike Adamson commented on CASSANDRA-19011:
--
I have reproduced this issue in a simple random test and have a fix. The issue
is as follows:
The PostingListRangeIterator.performSkipTo method currently looks like this:
{code:java}
protected void performSkipTo(PrimaryKey nextKey)
{
if (skipToToken != null && skipToToken.compareTo(nextKey) >= 0)
return;
skipToToken = nextKey;
needsSkipping = true;
}
{code}
The problem is in the first line. The comparison is >=. Currently, this means
that if the skipToToken is a token only primary key then it will be equal to
any other primary key with the same token. If nextKey is a wide key (with
clustering) it will match. This means that the skip will be to the first row in
the partition and not the required row indicated by the clustering.
The, easy, fix for this is to change the comparison to > so we only keep the
existing skipToToken if it is greater than nextKey. This fixes the problem but
I think it is worth looking at the PrimaryKey comparisons generally to make
sure that they meet a specific set of comparison requirements. This is the
second identified issue relating to PrimaryKey comparisons, the other being
CASSANDRA-19034.
> Harry-found exception in during SAI query
> -
>
> Key: CASSANDRA-19011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19011
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Alex Petrov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc
>
>
> Schema:
> {code:java}
> CREATE TABLE IF NOT EXISTS distributed_test_keyspace.tbl1 (pk1 bigint,ck1
> bigint,v1 ascii,v2 bigint, PRIMARY KEY (pk1, ck1)) WITH CLUSTERING ORDER BY
> (ck1 ASC);
> CREATE CUSTOM INDEX v1_sai_idx ON distributed_test_keyspace.tbl1 (v1) USING
> 'StorageAttachedIndex' WITH OPTIONS = {'case_sensitive': 'false',
> 'normalize': 'true', 'ascii': 'true'}; ;
> CREATE CUSTOM INDEX v2_sai_idx ON distributed_test_keyspace.tbl1 (v2) USING
> 'StorageAttachedIndex';
> {code}
> {code:java}
> java.lang.AssertionError: skipped to an item smaller than the target;
> iterator:
> org.apache.cassandra.index.sai.disk.IndexSearchResultIterator@f399f79, target
> key: PrimaryKey: { token: 8384965201802291970, partition:
> DecoratedKey(8384965201802291970, c4bc1c50f9e76a50), clustering:
> CLUSTERING:8b4b4c5991a4ea10 } , returned key: PrimaryKey: { token:
> 8384965201802291970, partition: DecoratedKey(8384965201802291970,
> c4bc1c50f9e76a50), clustering: CLUSTERING:89f1cf92658cb668 }
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:95)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:39)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.tryToComputeNext(AbstractGuavaIterator.java:122)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIterator.tryToComputeNext(KeyRangeIterator.java:129)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.hasNext(AbstractGuavaIterator.java:116)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKey(StorageAttachedIndexSearcher.java:274)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKeyInRange(StorageAttachedIndexSearcher.java:203)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextSelectedKeyInRange(StorageAttachedIndexSearcher.java:234)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextRowIterator(StorageAttachedIndexSearcher.java:188)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:169)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:111)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:91)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:338)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:201)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:186)
> at
>
[jira] [Commented] (CASSANDRA-19011) Harry-found exception in during SAI query
[
https://issues.apache.org/jira/browse/CASSANDRA-19011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790202#comment-17790202
]
Mike Adamson commented on CASSANDRA-19011:
--
[~ifesdjeen] Did you manage to isolate a reproduction for this issue? It's not
an easy one to reproduce but I will keep trying.
> Harry-found exception in during SAI query
> -
>
> Key: CASSANDRA-19011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19011
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Alex Petrov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc
>
>
> Schema:
> {code:java}
> CREATE TABLE IF NOT EXISTS distributed_test_keyspace.tbl1 (pk1 bigint,ck1
> bigint,v1 ascii,v2 bigint, PRIMARY KEY (pk1, ck1)) WITH CLUSTERING ORDER BY
> (ck1 ASC);
> CREATE CUSTOM INDEX v1_sai_idx ON distributed_test_keyspace.tbl1 (v1) USING
> 'StorageAttachedIndex' WITH OPTIONS = {'case_sensitive': 'false',
> 'normalize': 'true', 'ascii': 'true'}; ;
> CREATE CUSTOM INDEX v2_sai_idx ON distributed_test_keyspace.tbl1 (v2) USING
> 'StorageAttachedIndex';
> {code}
> {code:java}
> java.lang.AssertionError: skipped to an item smaller than the target;
> iterator:
> org.apache.cassandra.index.sai.disk.IndexSearchResultIterator@f399f79, target
> key: PrimaryKey: { token: 8384965201802291970, partition:
> DecoratedKey(8384965201802291970, c4bc1c50f9e76a50), clustering:
> CLUSTERING:8b4b4c5991a4ea10 } , returned key: PrimaryKey: { token:
> 8384965201802291970, partition: DecoratedKey(8384965201802291970,
> c4bc1c50f9e76a50), clustering: CLUSTERING:89f1cf92658cb668 }
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:95)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:39)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.tryToComputeNext(AbstractGuavaIterator.java:122)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIterator.tryToComputeNext(KeyRangeIterator.java:129)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.hasNext(AbstractGuavaIterator.java:116)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKey(StorageAttachedIndexSearcher.java:274)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKeyInRange(StorageAttachedIndexSearcher.java:203)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextSelectedKeyInRange(StorageAttachedIndexSearcher.java:234)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextRowIterator(StorageAttachedIndexSearcher.java:188)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:169)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:111)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:91)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:338)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:201)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:186)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:346)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:2186)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2581)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> relocated.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829) {code}
>
> Unfortunately, there's no tooling for shrinking around SAI just yet, but I
> have a programmatic repro using INSERT and DELETE statements. I will do my
> best to post it asap, but thought this can already be useful for visibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail:
[jira] [Updated] (CASSANDRA-19032) StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding is flakey
[ https://issues.apache.org/jira/browse/CASSANDRA-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19032: - Test and Documentation Plan: Current CI runs are in the comments Status: Patch Available (was: In Progress) > StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding is flakey > - > > Key: CASSANDRA-19032 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19032 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-rc > > Time Spent: 20m > Remaining Estimate: 0h > > The StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding has been > showing flakiness in test runs and need investigating and fixing. Looking at > the failure is seems that there is an edge condition where the index will > fail to complete its build if a table truncate is triggered at the right > moment. > We need to try and provoke this edge condition using injection and then see > what can be done to fix the build failure. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19032) StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding is flakey
[ https://issues.apache.org/jira/browse/CASSANDRA-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790096#comment-17790096 ] Mike Adamson edited comment on CASSANDRA-19032 at 11/27/23 2:33 PM: The patches and CI for this ticket are here: | [5.0|https://github.com/apache/cassandra/pull/2932]|[j11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/404/workflows/bae70e8f-7435-48d0-8651-8f03db19fa7d]|[j17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/404/workflows/5fdb25e1-6c63-4e53-9266-0b47703d4855]| |[trunk|https://github.com/apache/cassandra/pull/2932]|[j11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/405/workflows/bfb8801c-f432-47f5-9a6c-a810f1cbe67c]|[j17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/405/workflows/556278d4-19e8-4f79-841c-bb5bbc728cbf]| The patch provides a truncate task for SAI that marks the index as queryable. There was a window of opportunity if the initial index build and table truncate operations were triggered concurrently where the index build was interrupted. This meant that index wasn't marking itself as queryable. was (Author: mike_tr_adamson): The patches and CI for this ticket are here: | [5.0|https://github.com/apache/cassandra/pull/2932]|[j11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/404/workflows/bae70e8f-7435-48d0-8651-8f03db19fa7d]|[j17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/404/workflows/5fdb25e1-6c63-4e53-9266-0b47703d4855]| |[trunk|https://github.com/apache/cassandra/pull/2932]|[j11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/405/workflows/bfb8801c-f432-47f5-9a6c-a810f1cbe67c]|[j17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/405/workflows/556278d4-19e8-4f79-841c-bb5bbc728cbf]| > StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding is flakey > - > > Key: CASSANDRA-19032 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19032 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-rc > > Time Spent: 20m > Remaining Estimate: 0h > > The StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding has been > showing flakiness in test runs and need investigating and fixing. Looking at > the failure is seems that there is an edge condition where the index will > fail to complete its build if a table truncate is triggered at the right > moment. > We need to try and provoke this edge condition using injection and then see > what can be done to fix the build failure. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19032) StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding is flakey
[ https://issues.apache.org/jira/browse/CASSANDRA-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790096#comment-17790096 ] Mike Adamson commented on CASSANDRA-19032: -- The patches and CI for this ticket are here: | [5.0|https://github.com/apache/cassandra/pull/2932]|[j11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/404/workflows/bae70e8f-7435-48d0-8651-8f03db19fa7d]|[j17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/404/workflows/5fdb25e1-6c63-4e53-9266-0b47703d4855]| |[trunk|https://github.com/apache/cassandra/pull/2932]|[j11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/405/workflows/bfb8801c-f432-47f5-9a6c-a810f1cbe67c]|[j17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/405/workflows/556278d4-19e8-4f79-841c-bb5bbc728cbf]| > StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding is flakey > - > > Key: CASSANDRA-19032 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19032 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-rc > > Time Spent: 20m > Remaining Estimate: 0h > > The StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding has been > showing flakiness in test runs and need investigating and fixing. Looking at > the failure is seems that there is an edge condition where the index will > fail to complete its build if a table truncate is triggered at the right > moment. > We need to try and provoke this edge condition using injection and then see > what can be done to fix the build failure. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19011) Harry-found exception in during SAI query
[
https://issues.apache.org/jira/browse/CASSANDRA-19011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-19011:
-
Bug Category: Parent values: Availability(12983)
Complexity: Normal
Component/s: Feature/SAI
Discovered By: Fuzz Test
Severity: Normal
Status: Open (was: Triage Needed)
> Harry-found exception in during SAI query
> -
>
> Key: CASSANDRA-19011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19011
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Alex Petrov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc
>
>
> Schema:
> {code:java}
> CREATE TABLE IF NOT EXISTS distributed_test_keyspace.tbl1 (pk1 bigint,ck1
> bigint,v1 ascii,v2 bigint, PRIMARY KEY (pk1, ck1)) WITH CLUSTERING ORDER BY
> (ck1 ASC);
> CREATE CUSTOM INDEX v1_sai_idx ON distributed_test_keyspace.tbl1 (v1) USING
> 'StorageAttachedIndex' WITH OPTIONS = {'case_sensitive': 'false',
> 'normalize': 'true', 'ascii': 'true'}; ;
> CREATE CUSTOM INDEX v2_sai_idx ON distributed_test_keyspace.tbl1 (v2) USING
> 'StorageAttachedIndex';
> {code}
> {code:java}
> java.lang.AssertionError: skipped to an item smaller than the target;
> iterator:
> org.apache.cassandra.index.sai.disk.IndexSearchResultIterator@f399f79, target
> key: PrimaryKey: { token: 8384965201802291970, partition:
> DecoratedKey(8384965201802291970, c4bc1c50f9e76a50), clustering:
> CLUSTERING:8b4b4c5991a4ea10 } , returned key: PrimaryKey: { token:
> 8384965201802291970, partition: DecoratedKey(8384965201802291970,
> c4bc1c50f9e76a50), clustering: CLUSTERING:89f1cf92658cb668 }
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:95)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:39)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.tryToComputeNext(AbstractGuavaIterator.java:122)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIterator.tryToComputeNext(KeyRangeIterator.java:129)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.hasNext(AbstractGuavaIterator.java:116)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKey(StorageAttachedIndexSearcher.java:274)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKeyInRange(StorageAttachedIndexSearcher.java:203)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextSelectedKeyInRange(StorageAttachedIndexSearcher.java:234)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextRowIterator(StorageAttachedIndexSearcher.java:188)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:169)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:111)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:91)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:338)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:201)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:186)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:346)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:2186)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2581)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> relocated.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829) {code}
>
> Unfortunately, there's no tooling for shrinking around SAI just yet, but I
> have a programmatic repro using INSERT and DELETE statements. I will do my
> best to post it asap, but thought this can already be useful for visibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
[jira] [Assigned] (CASSANDRA-19011) Harry-found exception in during SAI query
[
https://issues.apache.org/jira/browse/CASSANDRA-19011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson reassigned CASSANDRA-19011:
Assignee: Mike Adamson (was: Caleb Rackliffe)
> Harry-found exception in during SAI query
> -
>
> Key: CASSANDRA-19011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19011
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alex Petrov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc
>
>
> Schema:
> {code:java}
> CREATE TABLE IF NOT EXISTS distributed_test_keyspace.tbl1 (pk1 bigint,ck1
> bigint,v1 ascii,v2 bigint, PRIMARY KEY (pk1, ck1)) WITH CLUSTERING ORDER BY
> (ck1 ASC);
> CREATE CUSTOM INDEX v1_sai_idx ON distributed_test_keyspace.tbl1 (v1) USING
> 'StorageAttachedIndex' WITH OPTIONS = {'case_sensitive': 'false',
> 'normalize': 'true', 'ascii': 'true'}; ;
> CREATE CUSTOM INDEX v2_sai_idx ON distributed_test_keyspace.tbl1 (v2) USING
> 'StorageAttachedIndex';
> {code}
> {code:java}
> java.lang.AssertionError: skipped to an item smaller than the target;
> iterator:
> org.apache.cassandra.index.sai.disk.IndexSearchResultIterator@f399f79, target
> key: PrimaryKey: { token: 8384965201802291970, partition:
> DecoratedKey(8384965201802291970, c4bc1c50f9e76a50), clustering:
> CLUSTERING:8b4b4c5991a4ea10 } , returned key: PrimaryKey: { token:
> 8384965201802291970, partition: DecoratedKey(8384965201802291970,
> c4bc1c50f9e76a50), clustering: CLUSTERING:89f1cf92658cb668 }
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:95)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIntersectionIterator.computeNext(KeyRangeIntersectionIterator.java:39)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.tryToComputeNext(AbstractGuavaIterator.java:122)
> at
> org.apache.cassandra.index.sai.iterators.KeyRangeIterator.tryToComputeNext(KeyRangeIterator.java:129)
> at
> org.apache.cassandra.utils.AbstractGuavaIterator.hasNext(AbstractGuavaIterator.java:116)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKey(StorageAttachedIndexSearcher.java:274)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextKeyInRange(StorageAttachedIndexSearcher.java:203)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextSelectedKeyInRange(StorageAttachedIndexSearcher.java:234)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.nextRowIterator(StorageAttachedIndexSearcher.java:188)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:169)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.computeNext(StorageAttachedIndexSearcher.java:111)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:91)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:338)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:201)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:186)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:346)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:2186)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2581)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> relocated.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829) {code}
>
> Unfortunately, there's no tooling for shrinking around SAI just yet, but I
> have a programmatic repro using INSERT and DELETE statements. I will do my
> best to post it asap, but thought this can already be useful for visibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19032) StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding is flakey
[ https://issues.apache.org/jira/browse/CASSANDRA-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19032: - Bug Category: Parent values: Availability(12983) Complexity: Normal Discovered By: Unit Test Severity: Normal Status: Open (was: Triage Needed) > StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding is flakey > - > > Key: CASSANDRA-19032 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19032 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-rc > > > The StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding has been > showing flakiness in test runs and need investigating and fixing. Looking at > the failure is seems that there is an edge condition where the index will > fail to complete its build if a table truncate is triggered at the right > moment. > We need to try and provoke this edge condition using injection and then see > what can be done to fix the build failure. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19032) StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding is flakey
[ https://issues.apache.org/jira/browse/CASSANDRA-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson reassigned CASSANDRA-19032: Assignee: Mike Adamson > StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding is flakey > - > > Key: CASSANDRA-19032 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19032 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-rc > > > The StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding has been > showing flakiness in test runs and need investigating and fixing. Looking at > the failure is seems that there is an edge condition where the index will > fail to complete its build if a table truncate is triggered at the right > moment. > We need to try and provoke this edge condition using injection and then see > what can be done to fix the build failure. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19034) SelectTest fails when run with SAI index
[
https://issues.apache.org/jira/browse/CASSANDRA-19034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789522#comment-17789522
]
Mike Adamson edited comment on CASSANDRA-19034 at 11/24/23 5:12 PM:
Here are patches for 5.0 and trunk:
|[5.0|https://github.com/apache/cassandra/pull/2929]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/401/workflows/6ab79f11-948d-4f95-b80e-6db662bb30fc]|
|[trunk|https://github.com/apache/cassandra/pull/2930]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/402/workflows/92aacb84-fd3a-48e0-9fb2-d1e2fe6fc71a]|
This fixes two specific issues in the code:
# Composite partition indexes were not working as expected because of an
optimisation we'd made during the CEP. We moved the check for the static row in
StorageAttachedIndexSearcher.applyIndexFilter to the top of the method before
the full partition check. This was wrong for composite partition indexes.
# Static primary keys were not being compared correctly against non-static
primary keys in the same partition. They should be equal but were being
compared as less than. This resulted in intersections between static and
non-static indexes never returning any rows.
was (Author: mike_tr_adamson):
Here are patches for 5.0 and trunk:
|[5.0|https://github.com/apache/cassandra/pull/2929]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/401/workflows/6ab79f11-948d-4f95-b80e-6db662bb30fc]|
|[trunk|https://github.com/apache/cassandra/pull/2930]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/402/workflows/92aacb84-fd3a-48e0-9fb2-d1e2fe6fc71a]|
> SelectTest fails when run with SAI index
>
>
> Key: CASSANDRA-19034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19034
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Branimir Lambov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When run with SAI index, the following two tests error out:
> {code}
> [junit-timeout] Testcase:
> testContainsKeyAndContainsWithIndexOnMapValue(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
>FAILED
> [junit-timeout] Got less rows than expected. Expected 1 but got 0
> [junit-timeout] junit.framework.AssertionFailedError: Got less rows than
> expected. Expected 1 but got 0
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1849)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testContainsKeyAndContainsWithIndexOnMapValue$9(SelectTest.java:625)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2238)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testContainsKeyAndContainsWithIndexOnMapValue(SelectTest.java:618)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Testcase:
> testFilterWithIndexForContains(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
> FAILED
> [junit-timeout] Invalid value for row 1 column 0 (k1 of type int), expected
> <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] junit.framework.AssertionFailedError: Invalid value for row 1
> column 0 (k1 of type int), expected <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1826)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testFilterWithIndexForContains$6(SelectTest.java:543)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2240)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testFilterWithIndexForContains(SelectTest.java:542)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
>
[jira] [Updated] (CASSANDRA-19034) SelectTest fails when run with SAI index
[
https://issues.apache.org/jira/browse/CASSANDRA-19034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-19034:
-
Test and Documentation Plan: Latest CI runs are in the comments
Status: Patch Available (was: In Progress)
> SelectTest fails when run with SAI index
>
>
> Key: CASSANDRA-19034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19034
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Branimir Lambov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When run with SAI index, the following two tests error out:
> {code}
> [junit-timeout] Testcase:
> testContainsKeyAndContainsWithIndexOnMapValue(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
>FAILED
> [junit-timeout] Got less rows than expected. Expected 1 but got 0
> [junit-timeout] junit.framework.AssertionFailedError: Got less rows than
> expected. Expected 1 but got 0
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1849)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testContainsKeyAndContainsWithIndexOnMapValue$9(SelectTest.java:625)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2238)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testContainsKeyAndContainsWithIndexOnMapValue(SelectTest.java:618)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Testcase:
> testFilterWithIndexForContains(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
> FAILED
> [junit-timeout] Invalid value for row 1 column 0 (k1 of type int), expected
> <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] junit.framework.AssertionFailedError: Invalid value for row 1
> column 0 (k1 of type int), expected <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1826)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testFilterWithIndexForContains$6(SelectTest.java:543)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2240)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testFilterWithIndexForContains(SelectTest.java:542)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The latter seems to be giving the results in the wrong order, and the order
> flips when the data is flushed.
> Caught during preparation of _latest config that would switch default to SAI
> (CASSANDRA-18753).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19034) SelectTest fails when run with SAI index
[
https://issues.apache.org/jira/browse/CASSANDRA-19034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789522#comment-17789522
]
Mike Adamson commented on CASSANDRA-19034:
--
Here are patches for 5.0 and trunk:
|[5.0|https://github.com/apache/cassandra/pull/2929]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/401/workflows/6ab79f11-948d-4f95-b80e-6db662bb30fc]|
|[trunk|https://github.com/apache/cassandra/pull/2930]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/402/workflows/92aacb84-fd3a-48e0-9fb2-d1e2fe6fc71a]|
> SelectTest fails when run with SAI index
>
>
> Key: CASSANDRA-19034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19034
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Branimir Lambov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When run with SAI index, the following two tests error out:
> {code}
> [junit-timeout] Testcase:
> testContainsKeyAndContainsWithIndexOnMapValue(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
>FAILED
> [junit-timeout] Got less rows than expected. Expected 1 but got 0
> [junit-timeout] junit.framework.AssertionFailedError: Got less rows than
> expected. Expected 1 but got 0
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1849)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testContainsKeyAndContainsWithIndexOnMapValue$9(SelectTest.java:625)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2238)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testContainsKeyAndContainsWithIndexOnMapValue(SelectTest.java:618)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Testcase:
> testFilterWithIndexForContains(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
> FAILED
> [junit-timeout] Invalid value for row 1 column 0 (k1 of type int), expected
> <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] junit.framework.AssertionFailedError: Invalid value for row 1
> column 0 (k1 of type int), expected <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1826)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testFilterWithIndexForContains$6(SelectTest.java:543)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2240)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testFilterWithIndexForContains(SelectTest.java:542)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The latter seems to be giving the results in the wrong order, and the order
> flips when the data is flushed.
> Caught during preparation of _latest config that would switch default to SAI
> (CASSANDRA-18753).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19034) SelectTest fails when run with SAI index
[
https://issues.apache.org/jira/browse/CASSANDRA-19034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788851#comment-17788851
]
Mike Adamson commented on CASSANDRA-19034:
--
[~adelapena] I'm happy with that. The more testing we can get round this the
better.
> SelectTest fails when run with SAI index
>
>
> Key: CASSANDRA-19034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19034
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Branimir Lambov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta
>
>
> When run with SAI index, the following two tests error out:
> {code}
> [junit-timeout] Testcase:
> testContainsKeyAndContainsWithIndexOnMapValue(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
>FAILED
> [junit-timeout] Got less rows than expected. Expected 1 but got 0
> [junit-timeout] junit.framework.AssertionFailedError: Got less rows than
> expected. Expected 1 but got 0
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1849)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testContainsKeyAndContainsWithIndexOnMapValue$9(SelectTest.java:625)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2238)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testContainsKeyAndContainsWithIndexOnMapValue(SelectTest.java:618)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Testcase:
> testFilterWithIndexForContains(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
> FAILED
> [junit-timeout] Invalid value for row 1 column 0 (k1 of type int), expected
> <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] junit.framework.AssertionFailedError: Invalid value for row 1
> column 0 (k1 of type int), expected <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1826)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testFilterWithIndexForContains$6(SelectTest.java:543)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2240)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testFilterWithIndexForContains(SelectTest.java:542)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The latter seems to be giving the results in the wrong order, and the order
> flips when the data is flushed.
> Caught during preparation of _latest config that would switch default to SAI
> (CASSANDRA-18753).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19034) SelectTest fails when run with SAI index
[
https://issues.apache.org/jira/browse/CASSANDRA-19034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788396#comment-17788396
]
Mike Adamson commented on CASSANDRA-19034:
--
I'd like to clarify what I think are the issues here:
# SelectTest.testFilterWithIndexForContains is failing because of an
optimisation make to the SAI query filtering. I have a fix for this currently
going onto the patch for this ticket.
# SelectTest.testContainsKeyAndContainsWithIndexOnMapValue is fixed on the
CASSANDRA-18166 patch.
# SecondaryIndexOnStaticColumnTest.testStaticIndexAndNonStaticIndex appears to
be a separate issue because it is not fixed by the fix for 1 but it does seem
related so I will investigate it on this ticket. If it is completely unrelated
then I'll create a new ticket for it. The only reason I think it may be related
is because it is related to static columns and that is part of the 1 issue.
> SelectTest fails when run with SAI index
>
>
> Key: CASSANDRA-19034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19034
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Branimir Lambov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta
>
>
> When run with SAI index, the following two tests error out:
> {code}
> [junit-timeout] Testcase:
> testContainsKeyAndContainsWithIndexOnMapValue(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
>FAILED
> [junit-timeout] Got less rows than expected. Expected 1 but got 0
> [junit-timeout] junit.framework.AssertionFailedError: Got less rows than
> expected. Expected 1 but got 0
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1849)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testContainsKeyAndContainsWithIndexOnMapValue$9(SelectTest.java:625)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2238)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testContainsKeyAndContainsWithIndexOnMapValue(SelectTest.java:618)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Testcase:
> testFilterWithIndexForContains(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
> FAILED
> [junit-timeout] Invalid value for row 1 column 0 (k1 of type int), expected
> <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] junit.framework.AssertionFailedError: Invalid value for row 1
> column 0 (k1 of type int), expected <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1826)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testFilterWithIndexForContains$6(SelectTest.java:543)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2240)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testFilterWithIndexForContains(SelectTest.java:542)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The latter seems to be giving the results in the wrong order, and the order
> flips when the data is flushed.
> Caught during preparation of _latest config that would switch default to SAI
> (CASSANDRA-18753).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19034) SelectTest fails when run with SAI index
[
https://issues.apache.org/jira/browse/CASSANDRA-19034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788387#comment-17788387
]
Mike Adamson commented on CASSANDRA-19034:
--
[~adelapena] Please note that this error has been fixed in CASSANDRA-18166
which is in review already.
> SelectTest fails when run with SAI index
>
>
> Key: CASSANDRA-19034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19034
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Branimir Lambov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta
>
>
> When run with SAI index, the following two tests error out:
> {code}
> [junit-timeout] Testcase:
> testContainsKeyAndContainsWithIndexOnMapValue(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
>FAILED
> [junit-timeout] Got less rows than expected. Expected 1 but got 0
> [junit-timeout] junit.framework.AssertionFailedError: Got less rows than
> expected. Expected 1 but got 0
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1849)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testContainsKeyAndContainsWithIndexOnMapValue$9(SelectTest.java:625)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2238)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testContainsKeyAndContainsWithIndexOnMapValue(SelectTest.java:618)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Testcase:
> testFilterWithIndexForContains(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
> FAILED
> [junit-timeout] Invalid value for row 1 column 0 (k1 of type int), expected
> <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] junit.framework.AssertionFailedError: Invalid value for row 1
> column 0 (k1 of type int), expected <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1826)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testFilterWithIndexForContains$6(SelectTest.java:543)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2240)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testFilterWithIndexForContains(SelectTest.java:542)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The latter seems to be giving the results in the wrong order, and the order
> flips when the data is flushed.
> Caught during preparation of _latest config that would switch default to SAI
> (CASSANDRA-18753).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19034) SelectTest fails when run with SAI index
[
https://issues.apache.org/jira/browse/CASSANDRA-19034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-19034:
-
Bug Category: Parent values: Correctness(12982)
Complexity: Normal
Discovered By: Unit Test
Severity: Normal
Status: Open (was: Triage Needed)
> SelectTest fails when run with SAI index
>
>
> Key: CASSANDRA-19034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19034
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Branimir Lambov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta
>
>
> When run with SAI index, the following two tests error out:
> {code}
> [junit-timeout] Testcase:
> testContainsKeyAndContainsWithIndexOnMapValue(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
>FAILED
> [junit-timeout] Got less rows than expected. Expected 1 but got 0
> [junit-timeout] junit.framework.AssertionFailedError: Got less rows than
> expected. Expected 1 but got 0
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1849)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testContainsKeyAndContainsWithIndexOnMapValue$9(SelectTest.java:625)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2238)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testContainsKeyAndContainsWithIndexOnMapValue(SelectTest.java:618)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Testcase:
> testFilterWithIndexForContains(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
> FAILED
> [junit-timeout] Invalid value for row 1 column 0 (k1 of type int), expected
> <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] junit.framework.AssertionFailedError: Invalid value for row 1
> column 0 (k1 of type int), expected <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1826)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testFilterWithIndexForContains$6(SelectTest.java:543)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2240)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testFilterWithIndexForContains(SelectTest.java:542)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The latter seems to be giving the results in the wrong order, and the order
> flips when the data is flushed.
> Caught during preparation of _latest config that would switch default to SAI
> (CASSANDRA-18753).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19034) SelectTest fails when run with SAI index
[
https://issues.apache.org/jira/browse/CASSANDRA-19034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson reassigned CASSANDRA-19034:
Assignee: Mike Adamson (was: Caleb Rackliffe)
> SelectTest fails when run with SAI index
>
>
> Key: CASSANDRA-19034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19034
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Branimir Lambov
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta
>
>
> When run with SAI index, the following two tests error out:
> {code}
> [junit-timeout] Testcase:
> testContainsKeyAndContainsWithIndexOnMapValue(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
>FAILED
> [junit-timeout] Got less rows than expected. Expected 1 but got 0
> [junit-timeout] junit.framework.AssertionFailedError: Got less rows than
> expected. Expected 1 but got 0
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1849)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testContainsKeyAndContainsWithIndexOnMapValue$9(SelectTest.java:625)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2238)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testContainsKeyAndContainsWithIndexOnMapValue(SelectTest.java:618)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Testcase:
> testFilterWithIndexForContains(org.apache.cassandra.cql3.validation.operations.SelectTest)-_jdk11:
> FAILED
> [junit-timeout] Invalid value for row 1 column 0 (k1 of type int), expected
> <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] junit.framework.AssertionFailedError: Invalid value for row 1
> column 0 (k1 of type int), expected <1> but got <0>
> [junit-timeout] Invalid value for row 1 column 2 (v of type set),
> expected <{4, 5, 6}> but got <{2, 3, 4}>
> [junit-timeout]
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1826)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.lambda$testFilterWithIndexForContains$6(SelectTest.java:543)
> [junit-timeout] at
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:2240)
> [junit-timeout] at
> org.apache.cassandra.cql3.validation.operations.SelectTest.testFilterWithIndexForContains(SelectTest.java:542)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The latter seems to be giving the results in the wrong order, and the order
> flips when the data is flushed.
> Caught during preparation of _latest config that would switch default to SAI
> (CASSANDRA-18753).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18927) Fix potential race condition in IndexViewManager during invalidation
[
https://issues.apache.org/jira/browse/CASSANDRA-18927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-18927:
-
Resolution: Fixed
Status: Resolved (was: Triage Needed)
This was fixed in CASSANDRA-18715 as part of CEP-30
> Fix potential race condition in IndexViewManager during invalidation
>
>
> Key: CASSANDRA-18927
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18927
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta
>
>
> There is a potential race condition in the {{IndexViewManager.invalidate}}
> method:
> {code:java}
> public void invalidate()
> {
> View currentView = view.get();
> for (SSTableIndex index : currentView)
> {
> index.markObsolete();
> }
> view.set(new View(context, Collections.emptyList()));
> } {code}
> We should {{getAndSet}} the view before marking the indexes as obsolete. This
> would avoid indexes potentially being made obsolete when being accessed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18166) Improve the code model around IndexContext
[ https://issues.apache.org/jira/browse/CASSANDRA-18166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18166: - Test and Documentation Plan: CircleCI runs are in the comments Status: Patch Available (was: Open) > Improve the code model around IndexContext > -- > > Key: CASSANDRA-18166 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18166 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Labels: SAI > Fix For: 5.0-rc > > Time Spent: 20m > Remaining Estimate: 0h > > We currently have a situation where we need to create an IndexContext that is > for a non-indexed column and therefore is never going to be used for indexing > or searching. This results in the IndexContext having to check for this at > points in the code with assertions. The reason for this that, even when the > column is non-indexed, we need to have information about the column for the > purpose of post-filtering. > It would make sense to split out the column / index information needed for > filtering from the indexing / searching requirements such that we could avoid > unnecessary assertions in the code. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18927) Fix potential race condition in IndexViewManager during invalidation
[
https://issues.apache.org/jira/browse/CASSANDRA-18927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson reassigned CASSANDRA-18927:
Assignee: Mike Adamson
> Fix potential race condition in IndexViewManager during invalidation
>
>
> Key: CASSANDRA-18927
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18927
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-rc
>
>
> There is a potential race condition in the {{IndexViewManager.invalidate}}
> method:
> {code:java}
> public void invalidate()
> {
> View currentView = view.get();
> for (SSTableIndex index : currentView)
> {
> index.markObsolete();
> }
> view.set(new View(context, Collections.emptyList()));
> } {code}
> We should {{getAndSet}} the view before marking the indexes as obsolete. This
> would avoid indexes potentially being made obsolete when being accessed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19032) StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding is flakey
Mike Adamson created CASSANDRA-19032: Summary: StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding is flakey Key: CASSANDRA-19032 URL: https://issues.apache.org/jira/browse/CASSANDRA-19032 Project: Cassandra Issue Type: Bug Components: Feature/SAI Reporter: Mike Adamson The StorageAttachedIndexDDLTest.concurrentTruncateWithIndexBuilding has been showing flakiness in test runs and need investigating and fixing. Looking at the failure is seems that there is an edge condition where the index will fail to complete its build if a table truncate is triggered at the right moment. We need to try and provoke this edge condition using injection and then see what can be done to fix the build failure. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18166) Improve the code model around IndexContext
[ https://issues.apache.org/jira/browse/CASSANDRA-18166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18166: - Change Category: Operability Complexity: Normal Status: Open (was: Triage Needed) > Improve the code model around IndexContext > -- > > Key: CASSANDRA-18166 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18166 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Labels: SAI > > We currently have a situation where we need to create an IndexContext that is > for a non-indexed column and therefore is never going to be used for indexing > or searching. This results in the IndexContext having to check for this at > points in the code with assertions. The reason for this that, even when the > column is non-indexed, we need to have information about the column for the > purpose of post-filtering. > It would make sense to split out the column / index information needed for > filtering from the indexing / searching requirements such that we could avoid > unnecessary assertions in the code. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18994) SAI range query does not play together with "IN"
[
https://issues.apache.org/jira/browse/CASSANDRA-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-18994:
-
Test and Documentation Plan: CI runs for this fix are in the comment section
Status: Patch Available (was: Open)
> SAI range query does not play together with "IN"
>
>
> Key: CASSANDRA-18994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18994
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Stefan Miklosovic
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta
>
>
> I am using schema from the website's quickstart.
> {code}
> cqlsh> DESCRIBE KEYSPACE cycling ;
> CREATE KEYSPACE cycling WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE cycling.cyclist_semi_pro (
> id int PRIMARY KEY,
> affiliation text,
> age int,
> country text,
> firstname text,
> lastname text,
> registration date
> ) WITH additional_write_policy = '99p'
> AND allow_auto_snapshot = true
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND cdc = false
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '16', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND memtable = 'default'
> AND crc_check_chance = 1.0
> AND default_time_to_live = 0
> AND extensions = {}
> AND gc_grace_seconds = 864000
> AND incremental_backups = true
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair = 'BLOCKING'
> AND speculative_retry = '99p';
> CREATE CUSTOM INDEX age_sai_idx ON cycling.cyclist_semi_pro (age) USING
> 'StorageAttachedIndex';
> CREATE CUSTOM INDEX country_sai_idx ON cycling.cyclist_semi_pro (country)
> USING 'StorageAttachedIndex' WITH OPTIONS = {'ascii': 'true',
> 'case_sensitive': 'false', 'normalize': 'true'};
> CREATE CUSTOM INDEX lastname_sai_idx ON cycling.cyclist_semi_pro (lastname)
> USING 'StorageAttachedIndex' WITH OPTIONS = {'ascii': 'true',
> 'case_sensitive': 'false', 'normalize': 'true'};
> CREATE CUSTOM INDEX registration_sai_idx ON cycling.cyclist_semi_pro
> (registration) USING 'StorageAttachedIndex';
> {code}
> Then I do:
> {code}
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE lastname in ('Cantona',
> 'Boyd');
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> execute this query as it might involve data filtering and thus may have
> unpredictable performance. If you want to execute this query despite the
> performance unpredictability, use ALLOW FILTERING"
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE lastname in ('Cantona',
> 'Boyd') ALLOW FILTERING;
> id | affiliation | age | country | firstname | lastname | registration
> +-+-+-+---+--+--
> 5 | Como Velocità | 24 | ITA | Irene | Cantona | 2012-07-22
> 20 | London Cyclists | 18 | GBR |Leslie | Boyd | 2012-12-15
> {code}
> But check this:
> {code}
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE registration >
> '2010-01-01' AND registration < '2015-12-31' and lastname in ('Cantona',
> 'Boyd') allow filtering;
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read]
> message="Operation failed - received 0 responses and 1 failures: UNKNOWN from
> localhost/127.0.0.1:7000" info={'consistency': 'ONE', 'required_responses':
> 1, 'received_responses': 0, 'failures': 1, 'error_code_map': {'127.0.0.1':
> '0x'}}
> {code}
> and in the logs:
> {code}
> java.lang.AssertionError: null
> at
> org.apache.cassandra.index.sai.plan.Expression.add(Expression.java:171)
> at
> org.apache.cassandra.index.sai.plan.Operation.buildIndexExpressions(Operation.java:136)
> at
> org.apache.cassandra.index.sai.plan.Operation$AndNode.analyze(Operation.java:303)
> at
> org.apache.cassandra.index.sai.plan.Operation$Node.doTreeAnalysis(Operation.java:266)
> at
> org.apache.cassandra.index.sai.plan.Operation$Node.analyzeTree(Operation.java:251)
> at
> org.apache.cassandra.index.sai.plan.Operation.buildIterator(Operation.java:185)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.(StorageAttachedIndexSearcher.java:151)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher.search(StorageAttachedIndexSearcher.java:107)
>
[jira] [Commented] (CASSANDRA-18994) SAI range query does not play together with "IN"
[
https://issues.apache.org/jira/browse/CASSANDRA-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783323#comment-17783323
]
Mike Adamson commented on CASSANDRA-18994:
--
Here are patches for 5.0 and trunk, along with CI runs:
|[5.0|https://github.com/apache/cassandra/pull/2864]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/364]|
|[trunk|https://github.com/apache/cassandra/pull/2865]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/365]|
The error was happening because the IN operator is not caught by any
restrictions and is allowed on indexed or non-indexed columns. SAI, currently,
does not support IN so it was hitting the assert at a point where we were
expecting a valid operator. Unfortunately, we have a situation, that will
hopefully be fixed by CASSANDRA-18166, where this code path can be used in a
situation where we don't have an index for the column that supports the
operator.
The fix has been to change the assert to include whether the index context
being used represents an indexed column. If it doesn't, then we can
legitimately continue and ignore the operator.
> SAI range query does not play together with "IN"
>
>
> Key: CASSANDRA-18994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18994
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Stefan Miklosovic
>Assignee: Mike Adamson
>Priority: Normal
> Fix For: 5.0-beta
>
>
> I am using schema from the website's quickstart.
> {code}
> cqlsh> DESCRIBE KEYSPACE cycling ;
> CREATE KEYSPACE cycling WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE cycling.cyclist_semi_pro (
> id int PRIMARY KEY,
> affiliation text,
> age int,
> country text,
> firstname text,
> lastname text,
> registration date
> ) WITH additional_write_policy = '99p'
> AND allow_auto_snapshot = true
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND cdc = false
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '16', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND memtable = 'default'
> AND crc_check_chance = 1.0
> AND default_time_to_live = 0
> AND extensions = {}
> AND gc_grace_seconds = 864000
> AND incremental_backups = true
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair = 'BLOCKING'
> AND speculative_retry = '99p';
> CREATE CUSTOM INDEX age_sai_idx ON cycling.cyclist_semi_pro (age) USING
> 'StorageAttachedIndex';
> CREATE CUSTOM INDEX country_sai_idx ON cycling.cyclist_semi_pro (country)
> USING 'StorageAttachedIndex' WITH OPTIONS = {'ascii': 'true',
> 'case_sensitive': 'false', 'normalize': 'true'};
> CREATE CUSTOM INDEX lastname_sai_idx ON cycling.cyclist_semi_pro (lastname)
> USING 'StorageAttachedIndex' WITH OPTIONS = {'ascii': 'true',
> 'case_sensitive': 'false', 'normalize': 'true'};
> CREATE CUSTOM INDEX registration_sai_idx ON cycling.cyclist_semi_pro
> (registration) USING 'StorageAttachedIndex';
> {code}
> Then I do:
> {code}
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE lastname in ('Cantona',
> 'Boyd');
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> execute this query as it might involve data filtering and thus may have
> unpredictable performance. If you want to execute this query despite the
> performance unpredictability, use ALLOW FILTERING"
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE lastname in ('Cantona',
> 'Boyd') ALLOW FILTERING;
> id | affiliation | age | country | firstname | lastname | registration
> +-+-+-+---+--+--
> 5 | Como Velocità | 24 | ITA | Irene | Cantona | 2012-07-22
> 20 | London Cyclists | 18 | GBR |Leslie | Boyd | 2012-12-15
> {code}
> But check this:
> {code}
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE registration >
> '2010-01-01' AND registration < '2015-12-31' and lastname in ('Cantona',
> 'Boyd') allow filtering;
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read]
> message="Operation failed - received 0 responses and 1 failures: UNKNOWN from
> localhost/127.0.0.1:7000" info={'consistency': 'ONE', 'required_responses':
> 1, 'received_responses': 0, 'failures': 1, 'error_code_map': {'127.0.0.1':
> '0x'}}
> {code}
> and in the logs:
> {code}
>
[jira] [Assigned] (CASSANDRA-18994) SAI range query does not play together with "IN"
[
https://issues.apache.org/jira/browse/CASSANDRA-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson reassigned CASSANDRA-18994:
Assignee: Mike Adamson
> SAI range query does not play together with "IN"
>
>
> Key: CASSANDRA-18994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18994
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Stefan Miklosovic
>Assignee: Mike Adamson
>Priority: Normal
>
> I am using schema from the website's quickstart.
> {code}
> cqlsh> DESCRIBE KEYSPACE cycling ;
> CREATE KEYSPACE cycling WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE cycling.cyclist_semi_pro (
> id int PRIMARY KEY,
> affiliation text,
> age int,
> country text,
> firstname text,
> lastname text,
> registration date
> ) WITH additional_write_policy = '99p'
> AND allow_auto_snapshot = true
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND cdc = false
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '16', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND memtable = 'default'
> AND crc_check_chance = 1.0
> AND default_time_to_live = 0
> AND extensions = {}
> AND gc_grace_seconds = 864000
> AND incremental_backups = true
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair = 'BLOCKING'
> AND speculative_retry = '99p';
> CREATE CUSTOM INDEX age_sai_idx ON cycling.cyclist_semi_pro (age) USING
> 'StorageAttachedIndex';
> CREATE CUSTOM INDEX country_sai_idx ON cycling.cyclist_semi_pro (country)
> USING 'StorageAttachedIndex' WITH OPTIONS = {'ascii': 'true',
> 'case_sensitive': 'false', 'normalize': 'true'};
> CREATE CUSTOM INDEX lastname_sai_idx ON cycling.cyclist_semi_pro (lastname)
> USING 'StorageAttachedIndex' WITH OPTIONS = {'ascii': 'true',
> 'case_sensitive': 'false', 'normalize': 'true'};
> CREATE CUSTOM INDEX registration_sai_idx ON cycling.cyclist_semi_pro
> (registration) USING 'StorageAttachedIndex';
> {code}
> Then I do:
> {code}
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE lastname in ('Cantona',
> 'Boyd');
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> execute this query as it might involve data filtering and thus may have
> unpredictable performance. If you want to execute this query despite the
> performance unpredictability, use ALLOW FILTERING"
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE lastname in ('Cantona',
> 'Boyd') ALLOW FILTERING;
> id | affiliation | age | country | firstname | lastname | registration
> +-+-+-+---+--+--
> 5 | Como Velocità | 24 | ITA | Irene | Cantona | 2012-07-22
> 20 | London Cyclists | 18 | GBR |Leslie | Boyd | 2012-12-15
> {code}
> But check this:
> {code}
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE registration >
> '2010-01-01' AND registration < '2015-12-31' and lastname in ('Cantona',
> 'Boyd') allow filtering;
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read]
> message="Operation failed - received 0 responses and 1 failures: UNKNOWN from
> localhost/127.0.0.1:7000" info={'consistency': 'ONE', 'required_responses':
> 1, 'received_responses': 0, 'failures': 1, 'error_code_map': {'127.0.0.1':
> '0x'}}
> {code}
> and in the logs:
> {code}
> java.lang.AssertionError: null
> at
> org.apache.cassandra.index.sai.plan.Expression.add(Expression.java:171)
> at
> org.apache.cassandra.index.sai.plan.Operation.buildIndexExpressions(Operation.java:136)
> at
> org.apache.cassandra.index.sai.plan.Operation$AndNode.analyze(Operation.java:303)
> at
> org.apache.cassandra.index.sai.plan.Operation$Node.doTreeAnalysis(Operation.java:266)
> at
> org.apache.cassandra.index.sai.plan.Operation$Node.analyzeTree(Operation.java:251)
> at
> org.apache.cassandra.index.sai.plan.Operation.buildIterator(Operation.java:185)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.(StorageAttachedIndexSearcher.java:151)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher.search(StorageAttachedIndexSearcher.java:107)
> at
> org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:431)
> at
>
[jira] [Updated] (CASSANDRA-18994) SAI range query does not play together with "IN"
[
https://issues.apache.org/jira/browse/CASSANDRA-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-18994:
-
Bug Category: Parent values: Correctness(12982)
Complexity: Low Hanging Fruit
Discovered By: Adhoc Test
Severity: Low
Status: Open (was: Triage Needed)
> SAI range query does not play together with "IN"
>
>
> Key: CASSANDRA-18994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18994
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Stefan Miklosovic
>Assignee: Mike Adamson
>Priority: Normal
>
> I am using schema from the website's quickstart.
> {code}
> cqlsh> DESCRIBE KEYSPACE cycling ;
> CREATE KEYSPACE cycling WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE cycling.cyclist_semi_pro (
> id int PRIMARY KEY,
> affiliation text,
> age int,
> country text,
> firstname text,
> lastname text,
> registration date
> ) WITH additional_write_policy = '99p'
> AND allow_auto_snapshot = true
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND cdc = false
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '16', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND memtable = 'default'
> AND crc_check_chance = 1.0
> AND default_time_to_live = 0
> AND extensions = {}
> AND gc_grace_seconds = 864000
> AND incremental_backups = true
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair = 'BLOCKING'
> AND speculative_retry = '99p';
> CREATE CUSTOM INDEX age_sai_idx ON cycling.cyclist_semi_pro (age) USING
> 'StorageAttachedIndex';
> CREATE CUSTOM INDEX country_sai_idx ON cycling.cyclist_semi_pro (country)
> USING 'StorageAttachedIndex' WITH OPTIONS = {'ascii': 'true',
> 'case_sensitive': 'false', 'normalize': 'true'};
> CREATE CUSTOM INDEX lastname_sai_idx ON cycling.cyclist_semi_pro (lastname)
> USING 'StorageAttachedIndex' WITH OPTIONS = {'ascii': 'true',
> 'case_sensitive': 'false', 'normalize': 'true'};
> CREATE CUSTOM INDEX registration_sai_idx ON cycling.cyclist_semi_pro
> (registration) USING 'StorageAttachedIndex';
> {code}
> Then I do:
> {code}
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE lastname in ('Cantona',
> 'Boyd');
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> execute this query as it might involve data filtering and thus may have
> unpredictable performance. If you want to execute this query despite the
> performance unpredictability, use ALLOW FILTERING"
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE lastname in ('Cantona',
> 'Boyd') ALLOW FILTERING;
> id | affiliation | age | country | firstname | lastname | registration
> +-+-+-+---+--+--
> 5 | Como Velocità | 24 | ITA | Irene | Cantona | 2012-07-22
> 20 | London Cyclists | 18 | GBR |Leslie | Boyd | 2012-12-15
> {code}
> But check this:
> {code}
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE registration >
> '2010-01-01' AND registration < '2015-12-31' and lastname in ('Cantona',
> 'Boyd') allow filtering;
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read]
> message="Operation failed - received 0 responses and 1 failures: UNKNOWN from
> localhost/127.0.0.1:7000" info={'consistency': 'ONE', 'required_responses':
> 1, 'received_responses': 0, 'failures': 1, 'error_code_map': {'127.0.0.1':
> '0x'}}
> {code}
> and in the logs:
> {code}
> java.lang.AssertionError: null
> at
> org.apache.cassandra.index.sai.plan.Expression.add(Expression.java:171)
> at
> org.apache.cassandra.index.sai.plan.Operation.buildIndexExpressions(Operation.java:136)
> at
> org.apache.cassandra.index.sai.plan.Operation$AndNode.analyze(Operation.java:303)
> at
> org.apache.cassandra.index.sai.plan.Operation$Node.doTreeAnalysis(Operation.java:266)
> at
> org.apache.cassandra.index.sai.plan.Operation$Node.analyzeTree(Operation.java:251)
> at
> org.apache.cassandra.index.sai.plan.Operation.buildIterator(Operation.java:185)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.(StorageAttachedIndexSearcher.java:151)
> at
>
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781092#comment-17781092 ] Mike Adamson commented on CASSANDRA-18715: -- [~adelapena] Yes, I would prefer that. I am actively working on the distributed test failures. I will have a look at the trunk failures as well. > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Attachments: signature.asc > > Time Spent: 25h 40m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780980#comment-17780980 ] Mike Adamson commented on CASSANDRA-18715: -- [~smiklosovic] I would prefer to merge this one because it is a failure for every run and put a separate patch for the rangeRestrictedTest. That is a random test failure and is going to take a little longer to fix. I am actively working on it, so hopefully not too long. > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Time Spent: 25h 40m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780972#comment-17780972 ] Mike Adamson edited comment on CASSANDRA-18715 at 10/30/23 11:42 AM: - PRs and CI for fixing VectorUpdateDeleteTest are here: |[5.0|https://github.com/apache/cassandra/pull/2850]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/357/workflows/46d6ab84-a2e9-4bcd-8a08-3ae2089476c9]| |[trunk|https://github.com/apache/cassandra/pull/2851]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/359/workflows/af539922-7531-447b-ae41-dd63cf0fe627]| was (Author: mike_tr_adamson): PRs and CI for fixing VectorUpdateDeleteTest are here: |[5.0|https://github.com/apache/cassandra/pull/2850]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/357/workflows/46d6ab84-a2e9-4bcd-8a08-3ae2089476c9]| > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Time Spent: 25h 40m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780972#comment-17780972 ] Mike Adamson commented on CASSANDRA-18715: -- PRs and CI for fixing VectorUpdateDeleteTest are here: |[5.0|https://github.com/apache/cassandra/pull/2850]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/357/workflows/46d6ab84-a2e9-4bcd-8a08-3ae2089476c9]| > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Time Spent: 25h 40m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780932#comment-17780932 ] Mike Adamson commented on CASSANDRA-18715: -- [~smiklosovic] [~mck] I am testing a patch for this. I will post the patch here with test runs when the test runs are complete. > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Time Spent: 25.5h > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779830#comment-17779830 ] Mike Adamson commented on CASSANDRA-18715: -- The contributors of the patch associated with this ticket wish to make it clear that the contribution adheres to clauses #5 & #7 of the Apache Foundation [ICLA|https://www.apache.org/licenses/icla.pdf]. > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta, 5.x > > Time Spent: 24h 20m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779562#comment-17779562 ] Mike Adamson commented on CASSANDRA-18715: -- The latest CI runs are here: |[5.0|https://github.com/apache/cassandra/pull/2673]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/348/workflows/982fd591-c53a-4aa6-9a95-68f10df6bfae]| |[trunk|https://github.com/apache/cassandra/pull/2765]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/349/workflows/20dc9c0d-695a-4f85-b2e4-a568bac06bc6]| > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta, 5.x > > Time Spent: 24h 10m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Description: The patch associated with this ticket adds a new vector index to SAI. This introduces the following new elements and changes to SAI: * VectorMemtableIndex - the in-memory representation of the vector indexes that writes data to a DiskANN instance * VectorSegmentBuilder - that writes a DiskANN graph to the following on-disk components: ** VECTOR - contains the floating point vectors associated with the graph ** TERMS - contains the HNSW graph on-disk representation written by a HnswGraphWriter ** POSTINGS - contains the index postings as written by a VectorPostingsWriter * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph was: The patch associated with this ticket adds a new vector index to SAI. This introduces the following new elements and changes to SAI: * VectorMemtableIndex - the in-memory representation of the vector indexes that writes data to a CassandraOnHeapHnsw instance * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk components: ** VECTOR - contains the floating point vectors associated with the graph ** TERMS - contains the HNSW graph on-disk representation written by a HnswGraphWriter ** POSTINGS - contains the index postings as written by a VectorPostingsWriter * VectorIndexSegmentSearcher - used to search the on-disk HNSW index > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta, 5.x > > Time Spent: 24h 10m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18905: - Status: Needs Committer (was: Review In Progress) > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0, 5.1 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776610#comment-17776610 ] Mike Adamson edited comment on CASSANDRA-18905 at 10/18/23 10:56 AM: - Latest test runs are here: |[5.0|https://github.com/apache/cassandra/pull/2769]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/331/workflows/f04324d8-33a2-47a8-b544-3296cf8b53d1]| |[trunk|https://github.com/apache/cassandra/pull/2771]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/332/workflows/946e28f4-2dec-4384-ac38-de011093f6c6]| The 5.0 unit test failure appears unrelated to these changes and broke on 11 & 17 test runs. It doesn't appear to be repeatable locally. was (Author: mike_tr_adamson): Latest test runs are here: |[5.0|https://github.com/apache/cassandra/pull/2769]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/331/workflows/f04324d8-33a2-47a8-b544-3296cf8b53d1]| |[trunk|https://github.com/apache/cassandra/pull/2771]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/332/workflows/946e28f4-2dec-4384-ac38-de011093f6c6]| > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0, 5.1 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776610#comment-17776610 ] Mike Adamson commented on CASSANDRA-18905: -- Latest test runs are here: |[5.0|https://github.com/apache/cassandra/pull/2769]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/331/workflows/f04324d8-33a2-47a8-b544-3296cf8b53d1]| |[trunk|https://github.com/apache/cassandra/pull/2771]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/332/workflows/946e28f4-2dec-4384-ac38-de011093f6c6]| > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0, 5.1 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18927) Fix potential race condition in IndexViewManager during invalidation
Mike Adamson created CASSANDRA-18927:
Summary: Fix potential race condition in IndexViewManager during
invalidation
Key: CASSANDRA-18927
URL: https://issues.apache.org/jira/browse/CASSANDRA-18927
Project: Cassandra
Issue Type: Bug
Components: Feature/SAI
Reporter: Mike Adamson
There is a potential race condition in the {{IndexViewManager.invalidate}}
method:
{code:java}
public void invalidate()
{
View currentView = view.get();
for (SSTableIndex index : currentView)
{
index.markObsolete();
}
view.set(new View(context, Collections.emptyList()));
} {code}
We should {{getAndSet}} the view before marking the indexes as obsolete. This
would avoid indexes potentially being made obsolete when being accessed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18926) SAI in-memory index does not include maximum term size check when adding terms
Mike Adamson created CASSANDRA-18926:
Summary: SAI in-memory index does not include maximum term size
check when adding terms
Key: CASSANDRA-18926
URL: https://issues.apache.org/jira/browse/CASSANDRA-18926
Project: Cassandra
Issue Type: Bug
Components: Feature/SAI
Reporter: Mike Adamson
The {{SSTableIndexWriter}} rejects terms that exceed a maximum term size with a
no-spam warning, but the {{TrieMemoryIndex}} does not do this.
We should check term sizes when rows are added and issue client warnings when
this happens. This still needs to happen in the {{SSTableIndexWriter}} to
handle terms during an initial index build.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18905: - Test and Documentation Plan: CircleCI builds are included in the first comment. Status: Patch Available (was: In Progress) > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771819#comment-17771819 ] Mike Adamson commented on CASSANDRA-18905: -- |[5.0|https://github.com/apache/cassandra/pull/2769]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/311/workflows/3fb0868b-a638-4582-9756-b799d4826b4d]| |[trunk|https://github.com/apache/cassandra/pull/2771]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/312/workflows/41e07357-e78b-4946-abd7-675033352753]| > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18905: - Status: Open (was: Triage Needed) > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson reassigned CASSANDRA-18905: Bug Category: Parent values: Availability(12983)Level 1 values: Process Crash(12992) Complexity: Low Hanging Fruit Discovered By: Performance Regression Test Fix Version/s: 5.0 Severity: Critical Assignee: Mike Adamson > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18473) Storage Attached Indexes (Phase 2)
[ https://issues.apache.org/jira/browse/CASSANDRA-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18473: - Change Category: Performance (was: Operability) > Storage Attached Indexes (Phase 2) > -- > > Key: CASSANDRA-18473 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18473 > Project: Cassandra > Issue Type: Epic > Components: Feature/2i Index >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Labels: SAI > > At the completion of CASSANDRA-16052, we should be able to release the core > capabilities of SAI in a stable, production-ready package. Once that begins > to gain traction, we'll be able to make improvements and add features for the > next major release. The major initial theme of this epic is likely to be > performance, but it will likely expand to include features like basic text > analysis, etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18473) Storage Attached Indexes (Phase 2)
[ https://issues.apache.org/jira/browse/CASSANDRA-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18473: - Change Category: Operability (was: Performance) > Storage Attached Indexes (Phase 2) > -- > > Key: CASSANDRA-18473 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18473 > Project: Cassandra > Issue Type: Epic > Components: Feature/2i Index >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Labels: SAI > > At the completion of CASSANDRA-16052, we should be able to release the core > capabilities of SAI in a stable, production-ready package. Once that begins > to gain traction, we'll be able to make improvements and add features for the > next major release. The major initial theme of this epic is likely to be > performance, but it will likely expand to include features like basic text > analysis, etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
Mike Adamson created CASSANDRA-18905: Summary: Index.Group is incorrectly unregistered from the SecondaryIndexManager Key: CASSANDRA-18905 URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 Project: Cassandra Issue Type: Bug Components: Feature/2i Index Reporter: Mike Adamson An Index.Group is removed from the SecondaryIndexManager during unregisterIndex if it contains no indexes after the index is unregistered. The code for removing the group uses the wrong key to remove the group from the indexGroups map. It is using the group object rather than the group name that is used as the key in the map. This means that the group is not added again if a new index is registered using that group. The knock on from this is that the StorageAttachedIndexGroup unregisters itself from the Tracker when it has no indexes after an index is removed. The same group with no tracker is then used for new indexes. This group then receives no notifications about sstable or memtable updates. The ultimate side effect of this is that, memtables are not released, resulting in memory leaks and indexes are not updated with new sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18892) Distributed tests can return ordering columns that have not been selected
[
https://issues.apache.org/jira/browse/CASSANDRA-18892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770459#comment-17770459
]
Mike Adamson commented on CASSANDRA-18892:
--
Patches and test runs are here:
|[4.0|https://github.com/apache/cassandra/pull/2753]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/301/workflows/49fae50f-5a55-4d7b-8af7-13b6d2251648]|
|[4.1|https://github.com/apache/cassandra/pull/2754]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/302/workflows/11e95c09-680b-4520-a3c4-b53223002ae7]|
|[5.0|https://github.com/apache/cassandra/pull/2750]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/298/workflows/9f9294d8-0025-438a-a93b-0f8f5a06a900]|
|[trunk|https://github.com/apache/cassandra/pull/2752]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/299/workflows/42674922-f4ca-4cb9-ad4e-d99d8f749a44]|
> Distributed tests can return ordering columns that have not been selected
> -
>
> Key: CASSANDRA-18892
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18892
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Normal
>
> The following test fails
> {code:java}
> @Test
> public void incorrectClusteringColumnTest() throws IOException
> {
> try (Cluster cluster = init(Cluster.build(1).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (k int, c int, v
> int, primary key(k, c))"));
> cluster.coordinator(1).execute(withKeyspace("INSERT INTO %s.t (k, c,
> v) VALUES (0, 1, 2)"), ConsistencyLevel.QUORUM);
> String query = withKeyspace("SELECT v FROM %s.t WHERE k IN (0, 1)
> ORDER BY c LIMIT 10");
> assertRows(cluster.coordinator(1).execute(query,
> ConsistencyLevel.ONE), row(2));
> }
> }
> {code}
> The query is returning the clustering column c as well as the regular column
> v.
> The reason for the extra column being returned is that the RowUtil is using
> ResultMessage.Rows.result.metadata.names instead on
> ResultMessage.Rows.result.metadata.requestNames(). This last method removes
> columns that have not been requested.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18892) Distributed tests can return ordering columns that have not been selected
[
https://issues.apache.org/jira/browse/CASSANDRA-18892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-18892:
-
Test and Documentation Plan:
This is fix the in-tree distributed test code so adds a single test to confirm
the fix.
The current test run against 5.0 is here:
https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/297/workflows/c41159b6-57f2-4bce-a238-68e52b43fc2c
Status: Patch Available (was: In Progress)
> Distributed tests can return ordering columns that have not been selected
> -
>
> Key: CASSANDRA-18892
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18892
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Normal
>
> The following test fails
> {code:java}
> @Test
> public void incorrectClusteringColumnTest() throws IOException
> {
> try (Cluster cluster = init(Cluster.build(1).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (k int, c int, v
> int, primary key(k, c))"));
> cluster.coordinator(1).execute(withKeyspace("INSERT INTO %s.t (k, c,
> v) VALUES (0, 1, 2)"), ConsistencyLevel.QUORUM);
> String query = withKeyspace("SELECT v FROM %s.t WHERE k IN (0, 1)
> ORDER BY c LIMIT 10");
> assertRows(cluster.coordinator(1).execute(query,
> ConsistencyLevel.ONE), row(2));
> }
> }
> {code}
> The query is returning the clustering column c as well as the regular column
> v.
> The reason for the extra column being returned is that the RowUtil is using
> ResultMessage.Rows.result.metadata.names instead on
> ResultMessage.Rows.result.metadata.requestNames(). This last method removes
> columns that have not been requested.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18892) Distributed tests can return ordering columns that have not been selected
[
https://issues.apache.org/jira/browse/CASSANDRA-18892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-18892:
-
Source Control Link: https://github.com/apache/cassandra/pull/2750
> Distributed tests can return ordering columns that have not been selected
> -
>
> Key: CASSANDRA-18892
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18892
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Normal
>
> The following test fails
> {code:java}
> @Test
> public void incorrectClusteringColumnTest() throws IOException
> {
> try (Cluster cluster = init(Cluster.build(1).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (k int, c int, v
> int, primary key(k, c))"));
> cluster.coordinator(1).execute(withKeyspace("INSERT INTO %s.t (k, c,
> v) VALUES (0, 1, 2)"), ConsistencyLevel.QUORUM);
> String query = withKeyspace("SELECT v FROM %s.t WHERE k IN (0, 1)
> ORDER BY c LIMIT 10");
> assertRows(cluster.coordinator(1).execute(query,
> ConsistencyLevel.ONE), row(2));
> }
> }
> {code}
> The query is returning the clustering column c as well as the regular column
> v.
> The reason for the extra column being returned is that the RowUtil is using
> ResultMessage.Rows.result.metadata.names instead on
> ResultMessage.Rows.result.metadata.requestNames(). This last method removes
> columns that have not been requested.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18892) Distributed tests can return ordering columns that have not been selected
[
https://issues.apache.org/jira/browse/CASSANDRA-18892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-18892:
-
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Complexity: Low Hanging Fruit
Discovered By: DTest
Severity: Low
Status: Open (was: Triage Needed)
> Distributed tests can return ordering columns that have not been selected
> -
>
> Key: CASSANDRA-18892
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18892
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Normal
>
> The following test fails
> {code:java}
> @Test
> public void incorrectClusteringColumnTest() throws IOException
> {
> try (Cluster cluster = init(Cluster.build(1).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (k int, c int, v
> int, primary key(k, c))"));
> cluster.coordinator(1).execute(withKeyspace("INSERT INTO %s.t (k, c,
> v) VALUES (0, 1, 2)"), ConsistencyLevel.QUORUM);
> String query = withKeyspace("SELECT v FROM %s.t WHERE k IN (0, 1)
> ORDER BY c LIMIT 10");
> assertRows(cluster.coordinator(1).execute(query,
> ConsistencyLevel.ONE), row(2));
> }
> }
> {code}
> The query is returning the clustering column c as well as the regular column
> v.
> The reason for the extra column being returned is that the RowUtil is using
> ResultMessage.Rows.result.metadata.names instead on
> ResultMessage.Rows.result.metadata.requestNames(). This last method removes
> columns that have not been requested.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18892) Distributed tests can return ordering columns that have not been selected
Mike Adamson created CASSANDRA-18892:
Summary: Distributed tests can return ordering columns that have
not been selected
Key: CASSANDRA-18892
URL: https://issues.apache.org/jira/browse/CASSANDRA-18892
Project: Cassandra
Issue Type: Bug
Components: Test/dtest/java
Reporter: Mike Adamson
Assignee: Mike Adamson
The following test fails
{code:java}
@Test
public void incorrectClusteringColumnTest() throws IOException
{
try (Cluster cluster = init(Cluster.build(1).start()))
{
cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (k int, c int, v
int, primary key(k, c))"));
cluster.coordinator(1).execute(withKeyspace("INSERT INTO %s.t (k, c, v)
VALUES (0, 1, 2)"), ConsistencyLevel.QUORUM);
String query = withKeyspace("SELECT v FROM %s.t WHERE k IN (0, 1) ORDER
BY c LIMIT 10");
assertRows(cluster.coordinator(1).execute(query, ConsistencyLevel.ONE),
row(2));
}
}
{code}
The query is returning the clustering column c as well as the regular column v.
The reason for the extra column being returned is that the RowUtil is using
ResultMessage.Rows.result.metadata.names instead on
ResultMessage.Rows.result.metadata.requestNames(). This last method removes
columns that have not been requested.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Test and Documentation Plan: Current test run: https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/288/workflows/e102cc8b-303a-400e-8265-f49cadf08eb5/jobs/19867 Status: Patch Available (was: In Progress) > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a CassandraOnHeapHnsw instance > * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk > components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk HNSW index -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Description: The patch associated with this ticket adds a new vector index to SAI. This introduces the following new elements and changes to SAI: * VectorMemtableIndex - the in-memory representation of the vector indexes that writes data to a CassandraOnHeapHnsw instance * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk components: ** VECTOR - contains the floating point vectors associated with the graph ** TERMS - contains the HNSW graph on-disk representation written by a HnswGraphWriter ** POSTINGS - contains the index postings as written by a VectorPostingsWriter * VectorIndexSegmentSearcher - used to search the on-disk HNSW index > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a CassandraOnHeapHnsw instance > * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk > components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk HNSW index -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
