[jira] [Commented] (CASSANDRA-9910) system_drop_column_family hangs on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-9910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644409#comment-14644409
]
Mike Adamson commented on CASSANDRA-9910:
-
This also affects {{truncate}} reproducible with:
{noformat}
create table test (k int, v int, primary key(k)) with compact storage;
client.truncate(test);
system_drop_column_family hangs on trunk
Key: CASSANDRA-9910
URL: https://issues.apache.org/jira/browse/CASSANDRA-9910
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Mike Adamson
Fix For: 3.0.0 rc1
{{system_drop_column_family}} is hanging on trunk. The code used to replicate
this is:
{noformat}
CfDef cfDef = new CfDef(ks, new_cf_2);
client.system_add_column_family(cfDef);
client.system_drop_column_family(cfDef.name);
{noformat}
The call is hanging in {{ColumnFamilyStore.switchMemtable}} because the
memtable flush is failing with
{noformat}
1 ERROR [MemtableFlushWriter:2] 2015-07-28 11:59:26,688 CassandraDaemon.java
(line 189) Exception in thread
Thread[MemtableFlushWriter:2,5,FailOnTimeoutGroup]
java.lang.AssertionError: null
at
org.apache.cassandra.db.PartitionColumns$Builder.add(PartitionColumns.java:148)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.cassandra.db.PartitionColumns$Builder.addAll(PartitionColumns.java:159)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.cassandra.db.Memtable$ColumnsCollector.get(Memtable.java:507)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:373)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:359)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
~[guava-16.0.1.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1034)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9854) Make CompactionController.maybeRefreshOverlaps public
[ https://issues.apache.org/jira/browse/CASSANDRA-9854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-9854: Fix Version/s: (was: 3.0.x) 3.0.0 rc1 Make CompactionController.maybeRefreshOverlaps public - Key: CASSANDRA-9854 URL: https://issues.apache.org/jira/browse/CASSANDRA-9854 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Mike Adamson Assignee: Mike Adamson Fix For: 3.0.0 rc1 Attachments: 9854.txt This method is currently package protected and means that 3rd party compaction strategies outside of the compaction package cannot use it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9937) logback-tools.xml is incorrectly configured for outputing to System.err
[
https://issues.apache.org/jira/browse/CASSANDRA-9937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-9937:
Attachment: 9937.txt
logback-tools.xml is incorrectly configured for outputing to System.err
---
Key: CASSANDRA-9937
URL: https://issues.apache.org/jira/browse/CASSANDRA-9937
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Mike Adamson
Assignee: Mike Adamson
Fix For: 2.1.9
Attachments: 9937.txt
{{logback-tools.xml}} is configured incorrectly in that it doesn't send
output to {{System.err}} as it appears to want to do.
{noformat}
appender name=STDERR target=System.err
class=ch.qos.logback.core.ConsoleAppender
{noformat}
should be
{noformat}
appender name=STDERR class=ch.qos.logback.core.ConsoleAppender
targetSystem.err/target
{noformat}
As a result output goes to {{System.out}}
Attaching a patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9937) logback-tools.xml is wrongly configured to output to System.err
Mike Adamson created CASSANDRA-9937:
---
Summary: logback-tools.xml is wrongly configured to output to
System.err
Key: CASSANDRA-9937
URL: https://issues.apache.org/jira/browse/CASSANDRA-9937
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Mike Adamson
Assignee: Mike Adamson
Fix For: 2.1.9
{{logback-tools.xml}} is configured incorrectly in that it doesn't send output
to {{System.err}} as it appears to want to do.
{noformat}
appender name=STDERR target=System.err
class=ch.qos.logback.core.ConsoleAppender
{noformat}
should be
{noformat}
appender name=STDERR class=ch.qos.logback.core.ConsoleAppender
targetSystem.err/target
{noformat}
As a result output goes to {{System.out}}
Attaching a patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9937) logback-tools.xml is incorrectly configured for outputing to System.err
[
https://issues.apache.org/jira/browse/CASSANDRA-9937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-9937:
Summary: logback-tools.xml is incorrectly configured for outputing to
System.err (was: logback-tools.xml is wrongly configured to output to
System.err)
logback-tools.xml is incorrectly configured for outputing to System.err
---
Key: CASSANDRA-9937
URL: https://issues.apache.org/jira/browse/CASSANDRA-9937
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Mike Adamson
Assignee: Mike Adamson
Fix For: 2.1.9
{{logback-tools.xml}} is configured incorrectly in that it doesn't send
output to {{System.err}} as it appears to want to do.
{noformat}
appender name=STDERR target=System.err
class=ch.qos.logback.core.ConsoleAppender
{noformat}
should be
{noformat}
appender name=STDERR class=ch.qos.logback.core.ConsoleAppender
targetSystem.err/target
{noformat}
As a result output goes to {{System.out}}
Attaching a patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9910) system_drop_column_family hangs on trunk
Mike Adamson created CASSANDRA-9910:
---
Summary: system_drop_column_family hangs on trunk
Key: CASSANDRA-9910
URL: https://issues.apache.org/jira/browse/CASSANDRA-9910
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Mike Adamson
Fix For: 3.0.0 rc1
{{system_drop_column_family}} is hanging on trunk. The code used to replicate
this is:
{noformat}
CfDef cfDef = new CfDef(ks, new_cf_2);
client.system_add_column_family(cfDef);
client.system_drop_column_family(cfDef.name);
{noformat}
The call is hanging in {{ColumnFamilyStore.switchMemtable}} because the
memtable flush is failing with
{noformat}
1 ERROR [MemtableFlushWriter:2] 2015-07-28 11:59:26,688 CassandraDaemon.java
(line 189) Exception in thread
Thread[MemtableFlushWriter:2,5,FailOnTimeoutGroup]
java.lang.AssertionError: null
at
org.apache.cassandra.db.PartitionColumns$Builder.add(PartitionColumns.java:148)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.cassandra.db.PartitionColumns$Builder.addAll(PartitionColumns.java:159)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.cassandra.db.Memtable$ColumnsCollector.get(Memtable.java:507)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:373)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:359)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
~[guava-16.0.1.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1034)
~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9916) batch_mutate failing on trunk
Mike Adamson created CASSANDRA-9916:
---
Summary: batch_mutate failing on trunk
Key: CASSANDRA-9916
URL: https://issues.apache.org/jira/browse/CASSANDRA-9916
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Mike Adamson
Fix For: 3.0.0 rc1
{{batch_mutate}} is failing on trunk with the following error:
{noformat}
java.lang.AssertionError: current =
ColumnDefinition{name=b@706172656e745f70617468,
type=org.apache.cassandra.db.marshal.BytesType, kind=STATIC,
componentIndex=null, indexName=cfs_parent_path, indexType=KEYS}, new =
ColumnDefinition{name=b@70617468,
type=org.apache.cassandra.db.marshal.BytesType, kind=STATIC,
componentIndex=null, indexName=cfs_path, indexType=KEYS}
at
org.apache.cassandra.db.rows.ArrayBackedRow$SortedBuilder.setColumn(ArrayBackedRow.java:617)
at
org.apache.cassandra.db.rows.ArrayBackedRow$SortedBuilder.addCell(ArrayBackedRow.java:630)
at
org.apache.cassandra.db.LegacyLayout$CellGrouper.addCell(LegacyLayout.java:891)
at
org.apache.cassandra.db.LegacyLayout$CellGrouper.addAtom(LegacyLayout.java:843)
at
org.apache.cassandra.db.LegacyLayout.getNextRow(LegacyLayout.java:390)
at
org.apache.cassandra.db.LegacyLayout.toUnfilteredRowIterator(LegacyLayout.java:326)
at
org.apache.cassandra.db.LegacyLayout.toUnfilteredRowIterator(LegacyLayout.java:288)
at
org.apache.cassandra.thrift.CassandraServer.createMutationList(CassandraServer.java:1110)
at
org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:1249)
{noformat}
The following mutations was passed to {{batch_mutate}} to get this error
{noformat}
mutationMap = {java.nio.HeapByteBuffer[pos=0 lim=32 cap=32]=
{inode=[
Mutation(column_or_supercolumn:ColumnOrSuperColumn(column:Column(name:80 62 00
04 70 61 74 68 00, value:2F, timestamp:1438165021749))),
Mutation(column_or_supercolumn:ColumnOrSuperColumn(column:Column(name:80 62 00
0B 70 61 72 65 6E 74 5F 70 61 74 68 00, value:6E 75 6C 6C,
timestamp:1438165021749))),
Mutation(column_or_supercolumn:ColumnOrSuperColumn(column:Column(name:80 62 00
08 73 65 6E 74 69 6E 65 6C 00, value:78, timestamp:1438165021749))),
Mutation(column_or_supercolumn:ColumnOrSuperColumn(column:Column(name:80 62 00
04 64 61 74 61 00, value:00 00 00 08 64 61 74 61 73 74 61 78 00 00 00 05 75 73
65 72 73 01 FF 00 00 00 00 00 04 00 00 00 01, timestamp:1438165021749)))
]}}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9854) Make CompactionController.maybeRefreshOverlaps public
[ https://issues.apache.org/jira/browse/CASSANDRA-9854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-9854: Attachment: 9854.txt Make CompactionController.maybeRefreshOverlaps public - Key: CASSANDRA-9854 URL: https://issues.apache.org/jira/browse/CASSANDRA-9854 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Mike Adamson Assignee: Mike Adamson Fix For: 3.0.x Attachments: 9854.txt This method is currently package protected and means that 3rd party compaction strategies outside of the compaction package cannot use it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-9854) Make CompactionController.maybeRefreshOverlaps public
Mike Adamson created CASSANDRA-9854: --- Summary: Make CompactionController.maybeRefreshOverlaps public Key: CASSANDRA-9854 URL: https://issues.apache.org/jira/browse/CASSANDRA-9854 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Mike Adamson Assignee: Mike Adamson Fix For: 3.0.x This method is currently package protected and means that 3rd party compaction strategies outside of the compaction package cannot use it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-10562) RolesCache should not be created for any authenticator that does not requireAuthentication
[
https://issues.apache.org/jira/browse/CASSANDRA-10562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson reassigned CASSANDRA-10562:
Assignee: Mike Adamson
> RolesCache should not be created for any authenticator that does not
> requireAuthentication
> --
>
> Key: CASSANDRA-10562
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10562
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.0
>
>
> The {{RolesCache.initCache}} method currently checks the authenticator as
> follows:
> {noformat}
> if (DatabaseDescriptor.getAuthenticator() instanceof
> AllowAllAuthenticator)
> return null;
> {noformat}
> This does not allow for any 3rd party authenticators that don't require
> authentication. It should check the {{requireAuthentication}} method on the
> authenticator to see if it should continue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10562) RolesCache should not be created for any authenticator that does not requireAuthentication
Mike Adamson created CASSANDRA-10562:
Summary: RolesCache should not be created for any authenticator
that does not requireAuthentication
Key: CASSANDRA-10562
URL: https://issues.apache.org/jira/browse/CASSANDRA-10562
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Mike Adamson
Fix For: 3.0.0
The {{RolesCache.initCache}} method currently checks the authenticator as
follows:
{noformat}
if (DatabaseDescriptor.getAuthenticator() instanceof
AllowAllAuthenticator)
return null;
{noformat}
This does not allow for any 3rd party authenticators that don't require
authentication. It should check the {{requireAuthentication}} method on the
authenticator to see if it should continue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10562) RolesCache should not be created for any authenticator that does not requireAuthentication
[
https://issues.apache.org/jira/browse/CASSANDRA-10562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-10562:
-
Attachment: 10562.txt
> RolesCache should not be created for any authenticator that does not
> requireAuthentication
> --
>
> Key: CASSANDRA-10562
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10562
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.0
>
> Attachments: 10562.txt
>
>
> The {{RolesCache.initCache}} method currently checks the authenticator as
> follows:
> {noformat}
> if (DatabaseDescriptor.getAuthenticator() instanceof
> AllowAllAuthenticator)
> return null;
> {noformat}
> This does not allow for any 3rd party authenticators that don't require
> authentication. It should check the {{requireAuthentication}} method on the
> authenticator to see if it should continue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10600) CqlInputFormat throws IOE if the size estimates are zero
[
https://issues.apache.org/jira/browse/CASSANDRA-10600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-10600:
-
Attachment: 10600.txt
> CqlInputFormat throws IOE if the size estimates are zero
>
>
> Key: CASSANDRA-10600
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10600
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.0
>
> Attachments: 10600.txt
>
>
> {{CqlInputFormat.describeSplits}} handles the case of no entry in the
> 'system.size_estimates' table but does not handle the situation when there is
> a zero size estimate in the table. This can happen if an input job is started
> immediately after data is added but before the {{SizeEstimatesRecorder}} has
> run.
> {{CqlInputFormat.describeSplits}} should handle 0 size estimate in the same
> manner as no size estimate and not sub-split but return the full split.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10600) CqlInputFormat throws IOE if the size estimates are zero
Mike Adamson created CASSANDRA-10600:
Summary: CqlInputFormat throws IOE if the size estimates are zero
Key: CASSANDRA-10600
URL: https://issues.apache.org/jira/browse/CASSANDRA-10600
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Mike Adamson
Assignee: Mike Adamson
Fix For: 3.0.0
{{CqlInputFormat.describeSplits}} handles the case of no entry in the
'system.size_estimates' table but does not handle the situation when there is a
zero size estimate in the table. This can happen if an input job is started
immediately after data is added but before the {{SizeEstimatesRecorder}} has
run.
{{CqlInputFormat.describeSplits}} should handle 0 size estimate in the same
manner as no size estimate and not sub-split but return the full split.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10608) Adding a dynamic column to a compact storage table with the same name as the partition key causes a memtable flush deadlock
[
https://issues.apache.org/jira/browse/CASSANDRA-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-10608:
-
Priority: Minor (was: Major)
> Adding a dynamic column to a compact storage table with the same name as the
> partition key causes a memtable flush deadlock
> ---
>
> Key: CASSANDRA-10608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10608
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.0
>
>
> The reproduction steps for this are as follows:
> # Create the following schema:
> {noformat}
> CREATE KEYSPACE ks WITH replication = { 'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE ks.cf (k int, v int, PRIMARY KEY(k)) WITH COMPACT STORAGE;
> {noformat}
> # Using the thrift client execute the following:
> {noformat}
> Column column = new Column(ByteBufferUtil.bytes("k"));
> column.setValue(ByteBufferUtil.bytes(1));
> column.setTimestamp(1);
> client.insert(key, new ColumnParent(compactCf), column,
> ConsistencyLevel.ONE);
> {noformat}
> This causes an invalid {{PartitionUpdate}} to be added to {{Memtable}}
> containing a {{PartitionColumns}} containing the partition key
> {{ColumnDefinition}}.
> This happens because {{LegacyLayout.decodeCellName}} does not check whether
> the {{ColumnDefinition}} is a partition key
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10608) Adding a dynamic column to a compact storage table with the same name as the partition key causes a memtable flush deadlock
Mike Adamson created CASSANDRA-10608:
Summary: Adding a dynamic column to a compact storage table with
the same name as the partition key causes a memtable flush deadlock
Key: CASSANDRA-10608
URL: https://issues.apache.org/jira/browse/CASSANDRA-10608
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Mike Adamson
Assignee: Mike Adamson
Fix For: 3.0.0
The reproduction steps for this are as follows:
# Create the following schema:
{noformat}
CREATE KEYSPACE ks WITH replication = { 'class': 'SimpleStrategy',
'replication_factor': '1'};
CREATE TABLE ks.cf (k int, v int, PRIMARY KEY(k)) WITH COMPACT STORAGE;
{noformat}
# Using the thrift client execute the following:
{noformat}
Column column = new Column(ByteBufferUtil.bytes("k"));
column.setValue(ByteBufferUtil.bytes(1));
column.setTimestamp(1);
client.insert(key, new ColumnParent(compactCf), column,
ConsistencyLevel.ONE);
{noformat}
This causes an invalid {{PartitionUpdate}} to be added to {{Memtable}}
containing a {{PartitionColumns}} containing the partition key
{{ColumnDefinition}}.
This happens because {{LegacyLayout.decodeCellName}} does not check whether the
{{ColumnDefinition}} is a partition key
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10608) Adding a dynamic column to a compact storage table with the same name as the partition key causes a memtable flush deadlock
[
https://issues.apache.org/jira/browse/CASSANDRA-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-10608:
-
Attachment: 10608.txt
> Adding a dynamic column to a compact storage table with the same name as the
> partition key causes a memtable flush deadlock
> ---
>
> Key: CASSANDRA-10608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10608
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: 10608.txt
>
>
> The reproduction steps for this are as follows:
> # Create the following schema:
> {noformat}
> CREATE KEYSPACE ks WITH replication = { 'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE ks.cf (k int, v int, PRIMARY KEY(k)) WITH COMPACT STORAGE;
> {noformat}
> # Using the thrift client execute the following:
> {noformat}
> Column column = new Column(ByteBufferUtil.bytes("k"));
> column.setValue(ByteBufferUtil.bytes(1));
> column.setTimestamp(1);
> client.insert(key, new ColumnParent(compactCf), column,
> ConsistencyLevel.ONE);
> {noformat}
> This causes an invalid {{PartitionUpdate}} to be added to {{Memtable}}
> containing a {{PartitionColumns}} containing the partition key
> {{ColumnDefinition}}.
> This happens because {{LegacyLayout.decodeCellName}} does not check whether
> the {{ColumnDefinition}} is a partition key
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10608) Adding a dynamic column to a compact storage table with the same name as the partition key causes a memtable flush deadlock
[
https://issues.apache.org/jira/browse/CASSANDRA-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980489#comment-14980489
]
Mike Adamson commented on CASSANDRA-10608:
--
[~slebresne] I have incorporated your suggestions in [~beobal]'s branch.
> Adding a dynamic column to a compact storage table with the same name as the
> partition key causes a memtable flush deadlock
> ---
>
> Key: CASSANDRA-10608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10608
> Project: Cassandra
> Issue Type: Bug
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: 10608.txt
>
>
> The reproduction steps for this are as follows:
> # Create the following schema:
> {noformat}
> CREATE KEYSPACE ks WITH replication = { 'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE ks.cf (k int, v int, PRIMARY KEY(k)) WITH COMPACT STORAGE;
> {noformat}
> # Using the thrift client execute the following:
> {noformat}
> Column column = new Column(ByteBufferUtil.bytes("k"));
> column.setValue(ByteBufferUtil.bytes(1));
> column.setTimestamp(1);
> client.insert(key, new ColumnParent(compactCf), column,
> ConsistencyLevel.ONE);
> {noformat}
> This causes an invalid {{PartitionUpdate}} to be added to {{Memtable}}
> containing a {{PartitionColumns}} containing the partition key
> {{ColumnDefinition}}.
> This happens because {{LegacyLayout.decodeCellName}} does not check whether
> the {{ColumnDefinition}} is a partition key
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10569) Keyspace validation errors are getting lost in system_add_keyspace
Mike Adamson created CASSANDRA-10569:
Summary: Keyspace validation errors are getting lost in
system_add_keyspace
Key: CASSANDRA-10569
URL: https://issues.apache.org/jira/browse/CASSANDRA-10569
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Mike Adamson
Fix For: 3.0.0
The following:
{noformat}
cassandraserver.system_add_keyspace(
new KsDef("xxx", SimpleStrategy.class.getSimpleName(),
Lists.newArrayList()));
{noformat}
used to throw an {{InvalidRequestException}} in 2.1.
In 3.0 the strategy validation has been removed from
{{KeyspaceMetadata.validate}} so the strategy errors don't get picked up until
the schema change has been announced. As a result the {{ConfigurationError}} is
swallowed in {{FBUtilities.waitOnFuture}} and thrown on as a
{{RuntimeException}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10569) Keyspace validation errors are getting lost in system_add_keyspace
[
https://issues.apache.org/jira/browse/CASSANDRA-10569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-10569:
-
Description:
The following:
{noformat}
cassandraserver.system_add_keyspace(
new KsDef("xxx", SimpleStrategy.class.getSimpleName(),
Lists.newArrayList()));
{noformat}
used to throw an {{InvalidRequestException}} in 2.1.
In 3.0 the strategy validation has been removed from
{{KeyspaceMetadata.validate}} so the strategy errors don't get picked up until
the schema change has been announced. As a result the {{ConfigurationError}} is
swallowed in {{FBUtilities.waitOnFuture}} and thrown on as a
{{RuntimeException}}.
This possibly affects {{system_update_keyspace}} as well.
was:
The following:
{noformat}
cassandraserver.system_add_keyspace(
new KsDef("xxx", SimpleStrategy.class.getSimpleName(),
Lists.newArrayList()));
{noformat}
used to throw an {{InvalidRequestException}} in 2.1.
In 3.0 the strategy validation has been removed from
{{KeyspaceMetadata.validate}} so the strategy errors don't get picked up until
the schema change has been announced. As a result the {{ConfigurationError}} is
swallowed in {{FBUtilities.waitOnFuture}} and thrown on as a
{{RuntimeException}}.
> Keyspace validation errors are getting lost in system_add_keyspace
> --
>
> Key: CASSANDRA-10569
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10569
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Mike Adamson
>Assignee: Sam Tunnicliffe
> Fix For: 3.0.0
>
>
> The following:
> {noformat}
> cassandraserver.system_add_keyspace(
> new KsDef("xxx", SimpleStrategy.class.getSimpleName(),
> Lists.newArrayList()));
> {noformat}
> used to throw an {{InvalidRequestException}} in 2.1.
> In 3.0 the strategy validation has been removed from
> {{KeyspaceMetadata.validate}} so the strategy errors don't get picked up
> until the schema change has been announced. As a result the
> {{ConfigurationError}} is swallowed in {{FBUtilities.waitOnFuture}} and
> thrown on as a {{RuntimeException}}.
> This possibly affects {{system_update_keyspace}} as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10717) Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
[
https://issues.apache.org/jira/browse/CASSANDRA-10717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15008567#comment-15008567
]
Mike Adamson commented on CASSANDRA-10717:
--
[~jlewandowski] I'm not that fussed because in your patch, if the
{{AuthProvider}} is set it will overwrite the {{PlainTextAuthProvider}} anyway.
The change I was proposing was to make this clearer in the code, that if the
user provided a username, password and {{AuthProvider}} then the provided
{{AuthProvider}} took precedence over the {{PlainTextAuthProvider}}.
> Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
> --
>
> Key: CASSANDRA-10717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10717
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
> Attachments: CqlConfigHelper_authentication_patch.patch
>
>
> Previously {{CqlInputFormat.getSplits(JobContext)}} method did not require
> anything but providing username and password in order to work with password
> protected Cassandra. Now, since CASSANDRA-9353 moved this part of the code
> from Thrift to Java Driver, the user needs to explicitly select
> {{AuthProvider}} to make it work.
> I propose a simple change in {{CqlConfigHelper}} which make
> {{CqlConfigHelper.getCluster}} method set credentials on {{Cluster.Builder}}
> if they are present in the configuration, prior to setting {{AuthProvider}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10608) Adding a dynamic column to a compact storage table with the same name as the partition key causes a memtable flush deadlock
[
https://issues.apache.org/jira/browse/CASSANDRA-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14985343#comment-14985343
]
Mike Adamson commented on CASSANDRA-10608:
--
[~slebresne] Apologies, I thought I'd rebased before the last run but obviously
not. I've kicked off the test builds again and am seeing 6 failures on the 3.0
branch. 4 of these are failing on the current 3.0 test build and the other 2
are successful here but seem to be failing on cassci.
> Adding a dynamic column to a compact storage table with the same name as the
> partition key causes a memtable flush deadlock
> ---
>
> Key: CASSANDRA-10608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10608
> Project: Cassandra
> Issue Type: Bug
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: 10608.txt
>
>
> The reproduction steps for this are as follows:
> # Create the following schema:
> {noformat}
> CREATE KEYSPACE ks WITH replication = { 'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE ks.cf (k int, v int, PRIMARY KEY(k)) WITH COMPACT STORAGE;
> {noformat}
> # Using the thrift client execute the following:
> {noformat}
> Column column = new Column(ByteBufferUtil.bytes("k"));
> column.setValue(ByteBufferUtil.bytes(1));
> column.setTimestamp(1);
> client.insert(key, new ColumnParent(compactCf), column,
> ConsistencyLevel.ONE);
> {noformat}
> This causes an invalid {{PartitionUpdate}} to be added to {{Memtable}}
> containing a {{PartitionColumns}} containing the partition key
> {{ColumnDefinition}}.
> This happens because {{LegacyLayout.decodeCellName}} does not check whether
> the {{ColumnDefinition}} is a partition key
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10608) Adding a dynamic column to a compact storage table with the same name as the partition key causes a memtable flush deadlock
[
https://issues.apache.org/jira/browse/CASSANDRA-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982765#comment-14982765
]
Mike Adamson commented on CASSANDRA-10608:
--
Good point, missed that. I've changed the branch logic in the inner branch to
your suggestion. Here is my branch with the fixes:
||3.0||trunk||
|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10608-3.0]|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10608-trunk]|
|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10608-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10608-trunk-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10608-3.0-dtest/]|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10608-trunk-dtest/]|
> Adding a dynamic column to a compact storage table with the same name as the
> partition key causes a memtable flush deadlock
> ---
>
> Key: CASSANDRA-10608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10608
> Project: Cassandra
> Issue Type: Bug
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: 10608.txt
>
>
> The reproduction steps for this are as follows:
> # Create the following schema:
> {noformat}
> CREATE KEYSPACE ks WITH replication = { 'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE ks.cf (k int, v int, PRIMARY KEY(k)) WITH COMPACT STORAGE;
> {noformat}
> # Using the thrift client execute the following:
> {noformat}
> Column column = new Column(ByteBufferUtil.bytes("k"));
> column.setValue(ByteBufferUtil.bytes(1));
> column.setTimestamp(1);
> client.insert(key, new ColumnParent(compactCf), column,
> ConsistencyLevel.ONE);
> {noformat}
> This causes an invalid {{PartitionUpdate}} to be added to {{Memtable}}
> containing a {{PartitionColumns}} containing the partition key
> {{ColumnDefinition}}.
> This happens because {{LegacyLayout.decodeCellName}} does not check whether
> the {{ColumnDefinition}} is a partition key
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10600) CqlInputFormat throws IOE if the size estimates are zero
[
https://issues.apache.org/jira/browse/CASSANDRA-10600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991810#comment-14991810
]
Mike Adamson commented on CASSANDRA-10600:
--
Here are the cassci test builds:
||3.0||trunk||
|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10600-3.0]|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10600-trunk]|
|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10600-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10600-trunk-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10600-3.0-dtest/]|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10600-trunk-dtest/]|
> CqlInputFormat throws IOE if the size estimates are zero
>
>
> Key: CASSANDRA-10600
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10600
> Project: Cassandra
> Issue Type: Bug
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.x, 3.0.x
>
> Attachments: 10600.txt
>
>
> {{CqlInputFormat.describeSplits}} handles the case of no entry in the
> 'system.size_estimates' table but does not handle the situation when there is
> a zero size estimate in the table. This can happen if an input job is started
> immediately after data is added but before the {{SizeEstimatesRecorder}} has
> run.
> {{CqlInputFormat.describeSplits}} should handle 0 size estimate in the same
> manner as no size estimate and not sub-split but return the full split.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9910) system_drop_column_family hangs on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-9910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991696#comment-14991696
]
Mike Adamson commented on CASSANDRA-9910:
-
This was being caused by CASSANDRA-10608.
> system_drop_column_family hangs on trunk
>
>
> Key: CASSANDRA-9910
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9910
> Project: Cassandra
> Issue Type: Bug
>Reporter: Mike Adamson
>Assignee: Branimir Lambov
>
> {{system_drop_column_family}} is hanging on trunk. The code used to replicate
> this is:
> {noformat}
> CfDef cfDef = new CfDef(ks, "new_cf_2");
> client.system_add_column_family(cfDef);
> client.system_drop_column_family(cfDef.name);
> {noformat}
> The call is hanging in {{ColumnFamilyStore.switchMemtable}} because the
> memtable flush is failing with
> {noformat}
> 1 ERROR [MemtableFlushWriter:2] 2015-07-28 11:59:26,688 CassandraDaemon.java
> (line 189) Exception in thread
> Thread[MemtableFlushWriter:2,5,FailOnTimeoutGroup]
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.PartitionColumns$Builder.add(PartitionColumns.java:148)
> ~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.cassandra.db.PartitionColumns$Builder.addAll(PartitionColumns.java:159)
> ~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.cassandra.db.Memtable$ColumnsCollector.get(Memtable.java:507)
> ~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:373)
> ~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:359)
> ~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
> ~[guava-16.0.1.jar:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1034)
> ~[cassandra-all-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10257) InvertedIndex trigger example has not been updated post 8099
[
https://issues.apache.org/jira/browse/CASSANDRA-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-10257:
-
Attachment: 10257.text
> InvertedIndex trigger example has not been updated post 8099
>
>
> Key: CASSANDRA-10257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10257
> Project: Cassandra
> Issue Type: Bug
> Components: Examples
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.x
>
> Attachments: 10257.text
>
>
> The {{InvertedIndex}} example is still using pre-8099 code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10257) InvertedIndex trigger example has not been updated post 8099
[
https://issues.apache.org/jira/browse/CASSANDRA-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-10257:
-
Attachment: (was: 10257.txt)
> InvertedIndex trigger example has not been updated post 8099
>
>
> Key: CASSANDRA-10257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10257
> Project: Cassandra
> Issue Type: Bug
> Components: Examples
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.x
>
> Attachments: 10257.txt
>
>
> The {{InvertedIndex}} example is still using pre-8099 code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10257) InvertedIndex trigger example has not been updated post 8099
[
https://issues.apache.org/jira/browse/CASSANDRA-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-10257:
-
Attachment: 10257.txt
> InvertedIndex trigger example has not been updated post 8099
>
>
> Key: CASSANDRA-10257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10257
> Project: Cassandra
> Issue Type: Bug
> Components: Examples
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.x
>
> Attachments: 10257.txt
>
>
> The {{InvertedIndex}} example is still using pre-8099 code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10257) InvertedIndex trigger example has not been updated post 8099
[
https://issues.apache.org/jira/browse/CASSANDRA-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-10257:
-
Attachment: 10257.txt
> InvertedIndex trigger example has not been updated post 8099
>
>
> Key: CASSANDRA-10257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10257
> Project: Cassandra
> Issue Type: Bug
> Components: Examples
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.x
>
> Attachments: 10257.txt
>
>
> The {{InvertedIndex}} example is still using pre-8099 code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10257) InvertedIndex trigger example has not been updated post 8099
[
https://issues.apache.org/jira/browse/CASSANDRA-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-10257:
-
Attachment: (was: 10257.text)
> InvertedIndex trigger example has not been updated post 8099
>
>
> Key: CASSANDRA-10257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10257
> Project: Cassandra
> Issue Type: Bug
> Components: Examples
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.x
>
>
> The {{InvertedIndex}} example is still using pre-8099 code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10731) Bootstrap starts before migration responses have completed
[
https://issues.apache.org/jira/browse/CASSANDRA-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-10731:
-
Reviewer: Sergio Bossa
> Bootstrap starts before migration responses have completed
> --
>
> Key: CASSANDRA-10731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10731
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> We have a number of failing tests running on slow VMs that are failing
> because {{MigrationManager.isReadyForBootstrap}} is return {{true}} when
> there are still inflight responses being processed to migration requests.
> This is happening because the {{MigrationTask.runMayThrow}} has completed but
> the {{IAsyncCallback.response}} is still running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10731) Bootstrap starts before migration responses have completed
[
https://issues.apache.org/jira/browse/CASSANDRA-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011615#comment-15011615
]
Mike Adamson commented on CASSANDRA-10731:
--
The proposed solution for this is as follows.
{{MigrationTask}} maintains a queue of {{CountdownLatch}} that complete as each
response callback completes. {{MigrationManager.isReadyForBootstrap}} returns
{{true}} if the queue is empty. If the queue is not empty then a new method
{{MigrationManager.waitTillReadyForBootstrap}} is called by {{StorageService}}.
This waits on each latch in the queue until they complete. The wait uses a
system property {{cassandra.migration_task_wait_in_seconds}} defined timeout
which defaults to 1 second.
||3.0||3.1||trunk||
|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10731-3.0]|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10731-3.1]|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10731-trunk]|
|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-3.1-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-trunk-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-3.0-dtest/]|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-3.1-dtest/]|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-trunk-dtest/]|
> Bootstrap starts before migration responses have completed
> --
>
> Key: CASSANDRA-10731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10731
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> We have a number of failing tests running on slow VMs that are failing
> because {{MigrationManager.isReadyForBootstrap}} is return {{true}} when
> there are still inflight responses being processed to migration requests.
> This is happening because the {{MigrationTask.runMayThrow}} has completed but
> the {{IAsyncCallback.response}} is still running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10731) Bootstrap starts before migration responses have completed
Mike Adamson created CASSANDRA-10731:
Summary: Bootstrap starts before migration responses have completed
Key: CASSANDRA-10731
URL: https://issues.apache.org/jira/browse/CASSANDRA-10731
Project: Cassandra
Issue Type: Bug
Components: Streaming and Messaging
Reporter: Mike Adamson
Assignee: Mike Adamson
Fix For: 3.0.x
We have a number of failing tests running on slow VMs that are failing because
{{MigrationManager.isReadyForBootstrap}} is return {{true}} when there are
still inflight responses being processed to migration requests. This is
happening because the {{MigrationTask.runMayThrow}} has completed but the
{{IAsyncCallback.response}} is still running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10717) Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
[
https://issues.apache.org/jira/browse/CASSANDRA-10717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15008332#comment-15008332
]
Mike Adamson commented on CASSANDRA-10717:
--
[~jlewandowski] Wouldn't it make more sense to have the check for the
authProvider first and then the username & password? Something like this:
{noformat}
if (authProvider.isPresent())
builder.withAuthProvider(authProvider.get());
else if (username.isPresent() && password.isPresent())
builder.withCredentials(username.get(), password.get());
{noformat}
This would allow for a user setting the username and password along with a
non-standard auth provider.
> Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
> --
>
> Key: CASSANDRA-10717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10717
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
> Attachments: CqlConfigHelper_authentication_patch.patch
>
>
> Previously {{CqlInputFormat.getSplits(JobContext)}} method did not require
> anything but providing username and password in order to work with password
> protected Cassandra. Now, since CASSANDRA-9353 moved this part of the code
> from Thrift to Java Driver, the user needs to explicitly select
> {{AuthProvider}} to make it work.
> I propose a simple change in {{CqlConfigHelper}} which make
> {{CqlConfigHelper.getCluster}} method set credentials on {{Cluster.Builder}}
> if they are present in the configuration, prior to setting {{AuthProvider}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10257) InvertedIndex trigger example has not been updated post 8099
Mike Adamson created CASSANDRA-10257:
Summary: InvertedIndex trigger example has not been updated post
8099
Key: CASSANDRA-10257
URL: https://issues.apache.org/jira/browse/CASSANDRA-10257
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Mike Adamson
Fix For: 3.0 beta 2
The {{InvertedIndex}} example is still using pre-8099 code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10257) InvertedIndex trigger example has not been updated post 8099
[
https://issues.apache.org/jira/browse/CASSANDRA-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939663#comment-14939663
]
Mike Adamson commented on CASSANDRA-10257:
--
[~iamaleksey] I've done a search and can't find anyone who has successfully
managed to get the {{InvertedIndex}} example working. The instructions in the
README are not intuitive and although it is easy to install and build it is not
easy to get it to do what it is meant to do.
Would you object to removing this example and adding a simpler and better
documented example that does something like adding entries to an audit table?
> InvertedIndex trigger example has not been updated post 8099
>
>
> Key: CASSANDRA-10257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10257
> Project: Cassandra
> Issue Type: Bug
> Components: Examples
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.x
>
>
> The {{InvertedIndex}} example is still using pre-8099 code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10257) InvertedIndex trigger example has not been updated post 8099
[
https://issues.apache.org/jira/browse/CASSANDRA-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940962#comment-14940962
]
Mike Adamson commented on CASSANDRA-10257:
--
Yes, I'll try and get a patch together in the next couple of days.
> InvertedIndex trigger example has not been updated post 8099
>
>
> Key: CASSANDRA-10257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10257
> Project: Cassandra
> Issue Type: Bug
> Components: Examples
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.x
>
>
> The {{InvertedIndex}} example is still using pre-8099 code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10817) DROP USER is not case-sensitive
Mike Adamson created CASSANDRA-10817:
Summary: DROP USER is not case-sensitive
Key: CASSANDRA-10817
URL: https://issues.apache.org/jira/browse/CASSANDRA-10817
Project: Cassandra
Issue Type: Bug
Components: CQL
Reporter: Mike Adamson
Priority: Minor
Fix For: 2.2.4
As per the summary {{DROP USER}} is not case sensitive, so:
{noformat}
CREATE USER 'Test';
LIST USERS;
name | super
---+---
Test | False
cassandra | True
DROP USER 'Test';
InvalidRequest: code=2200 [Invalid query] message="test doesn't exist"
{noformat}
{{DROP ROLE}} is case-sensitive and will drop the above user.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10852) Add requireAuthorization method to IAuthorizer
[
https://issues.apache.org/jira/browse/CASSANDRA-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056182#comment-15056182
]
Mike Adamson commented on CASSANDRA-10852:
--
Good point. I've updated the branch to use a default implementation of the
method.
> Add requireAuthorization method to IAuthorizer
> --
>
> Key: CASSANDRA-10852
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10852
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.2
>
>
> The {{IAuthenticator}} interface has a {{requireAuthentication}} to indicate
> whether an implementation requires an explicit login. For consistency it
> would be useful for 3rd party implementers if the {{IAuthorizer}} had a
> similar method {{requireAuthorization}} that would indicate if the
> implementation required explicit authorization.
> This would mean that we could remove and explicit {{instanceof}} checks for
> {{AllowAllAuthenticator}} and {{AllowAllAuthorizer}} in the code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10852) Add requireAuthorization method to IAuthorizer
[
https://issues.apache.org/jira/browse/CASSANDRA-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054365#comment-15054365
]
Mike Adamson commented on CASSANDRA-10852:
--
Branch and tests:
||trunk||
|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10852-trunk]|
|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10852-trunk-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10852-trunk-dtest/]|
> Add requireAuthorization method to IAuthorizer
> --
>
> Key: CASSANDRA-10852
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10852
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.2
>
>
> The {{IAuthenticator}} interface has a {{requireAuthentication}} to indicate
> whether an implementation requires an explicit login. For consistency it
> would be useful for 3rd party implementers if the {{IAuthorizer}} had a
> similar method {{requireAuthorization}} that would indicate if the
> implementation required explicit authorization.
> This would mean that we could remove and explicit {{instanceof}} checks for
> {{AllowAllAuthenticator}} and {{AllowAllAuthorizer}} in the code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10852) Add requireAuthorization method to IAuthorizer
Mike Adamson created CASSANDRA-10852:
Summary: Add requireAuthorization method to IAuthorizer
Key: CASSANDRA-10852
URL: https://issues.apache.org/jira/browse/CASSANDRA-10852
Project: Cassandra
Issue Type: New Feature
Reporter: Mike Adamson
Assignee: Mike Adamson
Priority: Minor
Fix For: 3.2
The {{IAuthenticator}} interface has a {{requireAuthentication}} to indicate
whether an implementation requires an explicit login. For consistency it would
be useful for 3rd party implementers if the {{IAuthorizer}} had a similar
method {{requireAuthorization}} that would indicate if the implementation
required explicit authorization.
This would mean that we could remove and explicit {{instanceof}} checks for
{{AllowAllAuthenticator}} and {{AllowAllAuthorizer}} in the code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10873) Allow sstableloader to work with 3rd party authentication providers
[
https://issues.apache.org/jira/browse/CASSANDRA-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063812#comment-15063812
]
Mike Adamson commented on CASSANDRA-10873:
--
I have updated both branches with your changes.
> Allow sstableloader to work with 3rd party authentication providers
> ---
>
> Key: CASSANDRA-10873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10873
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> When sstableloader was changed to use native protocol instead of thrift there
> was a regression in that now sstableloader (BulkLoader) only takes
> {{username}} and {{password}} as credentials so only works with the
> {{PlainTextAuthProvider}} provided by the java driver.
> Previously it allowed 3rd party auth providers to be used, we need to add
> back that ability by allowing the full classname of the auth provider to be
> passed as a parameter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10873) Allow sstableloader to work with 3rd party authentication providers
Mike Adamson created CASSANDRA-10873:
Summary: Allow sstableloader to work with 3rd party authentication
providers
Key: CASSANDRA-10873
URL: https://issues.apache.org/jira/browse/CASSANDRA-10873
Project: Cassandra
Issue Type: New Feature
Components: Tools
Reporter: Mike Adamson
Assignee: Mike Adamson
Fix For: 3.2
sstableloader (BulkLoader) currently only takes {{username}} and {{password}}
as credentials so only works with the {{PlainTextAuthProvider}} provided by the
java driver.
It should allow 3rd party auth providers to be used by allowing the full
classname of the auth provider to be passed as a parameter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10873) Allow sstableloader to work with 3rd party authentication providers
[
https://issues.apache.org/jira/browse/CASSANDRA-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-10873:
-
Fix Version/s: (was: 3.2)
3.0.x
> Allow sstableloader to work with 3rd party authentication providers
> ---
>
> Key: CASSANDRA-10873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10873
> Project: Cassandra
> Issue Type: New Feature
> Components: Tools
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> sstableloader (BulkLoader) currently only takes {{username}} and {{password}}
> as credentials so only works with the {{PlainTextAuthProvider}} provided by
> the java driver.
> It should allow 3rd party auth providers to be used by allowing the full
> classname of the auth provider to be passed as a parameter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10873) Allow sstableloader to work with 3rd party authentication providers
[
https://issues.apache.org/jira/browse/CASSANDRA-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15059819#comment-15059819
]
Mike Adamson commented on CASSANDRA-10873:
--
Patch for this here:
||3.0||trunk||
|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10873-3.0]|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10873-trunk]|
|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10873-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10873-trunk-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10873-3.0-dtest/]|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10873-trunk-dtest/]|
The bulkloader now takes the {{-ap }} parameter to
allow a 3rd party auth provider to be used. This can be combined with the {{-u
}} and {{-pw }} parameters if the auth provider supports
plain text credentials. In this case the auth provider must have a constructor
of the form:
{noformat}
public MyAuthProvider(String username, String password)
{
...
}
{noformat}
> Allow sstableloader to work with 3rd party authentication providers
> ---
>
> Key: CASSANDRA-10873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10873
> Project: Cassandra
> Issue Type: New Feature
> Components: Tools
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> sstableloader (BulkLoader) currently only takes {{username}} and {{password}}
> as credentials so only works with the {{PlainTextAuthProvider}} provided by
> the java driver.
> It should allow 3rd party auth providers to be used by allowing the full
> classname of the auth provider to be passed as a parameter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10731) Bootstrap starts before migration responses have completed
[
https://issues.apache.org/jira/browse/CASSANDRA-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011615#comment-15011615
]
Mike Adamson edited comment on CASSANDRA-10731 at 11/20/15 10:54 AM:
-
The proposed solution for this is as follows.
{{MigrationTask}} maintains a queue of {{CountdownLatch}} that complete as each
response callback completes. {{MigrationManager.isReadyForBootstrap}} returns
{{true}} if the queue is empty. If the queue is not empty then a new method
{{MigrationManager.waitTillReadyForBootstrap}} is called by {{StorageService}}.
This waits on each latch in the queue until they complete. The wait uses a
system property {{cassandra.migration_task_wait_in_seconds}} defined timeout
which defaults to 1 second.
||3.0||trunk||
|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10731-3.0]|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10731-trunk]|
|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-trunk-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-3.0-dtest/]|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-trunk-dtest/]|
was (Author: mike_tr_adamson):
The proposed solution for this is as follows.
{{MigrationTask}} maintains a queue of {{CountdownLatch}} that complete as each
response callback completes. {{MigrationManager.isReadyForBootstrap}} returns
{{true}} if the queue is empty. If the queue is not empty then a new method
{{MigrationManager.waitTillReadyForBootstrap}} is called by {{StorageService}}.
This waits on each latch in the queue until they complete. The wait uses a
system property {{cassandra.migration_task_wait_in_seconds}} defined timeout
which defaults to 1 second.
||3.0||3.1||trunk||
|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10731-3.0]|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10731-3.1]|[branch|https://github.com/mike-tr-adamson/cassandra/tree/10731-trunk]|
|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-3.1-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-trunk-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-3.0-dtest/]|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-3.1-dtest/]|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-10731-trunk-dtest/]|
> Bootstrap starts before migration responses have completed
> --
>
> Key: CASSANDRA-10731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10731
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> We have a number of failing tests running on slow VMs that are failing
> because {{MigrationManager.isReadyForBootstrap}} is return {{true}} when
> there are still inflight responses being processed to migration requests.
> This is happening because the {{MigrationTask.runMayThrow}} has completed but
> the {{IAsyncCallback.response}} is still running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10731) Bootstrap starts before migration responses have completed
[
https://issues.apache.org/jira/browse/CASSANDRA-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15015595#comment-15015595
]
Mike Adamson commented on CASSANDRA-10731:
--
Ok, I've removed the fail hard on the inflight check because it was breaking
the ability to resume bootstrap if the node being bootstrapped from goes down.
> Bootstrap starts before migration responses have completed
> --
>
> Key: CASSANDRA-10731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10731
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> We have a number of failing tests running on slow VMs that are failing
> because {{MigrationManager.isReadyForBootstrap}} is return {{true}} when
> there are still inflight responses being processed to migration requests.
> This is happening because the {{MigrationTask.runMayThrow}} has completed but
> the {{IAsyncCallback.response}} is still running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10717) Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
[
https://issues.apache.org/jira/browse/CASSANDRA-10717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15015553#comment-15015553
]
Mike Adamson commented on CASSANDRA-10717:
--
+1 LGTM
> Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
> --
>
> Key: CASSANDRA-10717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10717
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
> Attachments: CqlConfigHelper_authentication_patch-2.patch,
> CqlConfigHelper_authentication_patch.patch
>
>
> Previously {{CqlInputFormat.getSplits(JobContext)}} method did not require
> anything but providing username and password in order to work with password
> protected Cassandra. Now, since CASSANDRA-9353 moved this part of the code
> from Thrift to Java Driver, the user needs to explicitly select
> {{AuthProvider}} to make it work.
> I propose a simple change in {{CqlConfigHelper}} which make
> {{CqlConfigHelper.getCluster}} method set credentials on {{Cluster.Builder}}
> if they are present in the configuration, prior to setting {{AuthProvider}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10731) Bootstrap starts before migration responses have completed
[
https://issues.apache.org/jira/browse/CASSANDRA-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15018377#comment-15018377
]
Mike Adamson commented on CASSANDRA-10731:
--
I have pushed the branches again with the above change to propagate the
interrupt status.
> Bootstrap starts before migration responses have completed
> --
>
> Key: CASSANDRA-10731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10731
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> We have a number of failing tests running on slow VMs that are failing
> because {{MigrationManager.isReadyForBootstrap}} is return {{true}} when
> there are still inflight responses being processed to migration requests.
> This is happening because the {{MigrationTask.runMayThrow}} has completed but
> the {{IAsyncCallback.response}} is still running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10731) Bootstrap starts before migration responses have completed
[
https://issues.apache.org/jira/browse/CASSANDRA-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013349#comment-15013349
]
Mike Adamson commented on CASSANDRA-10731:
--
[~jjordan] Probably, yes. I've changed the
{{MigrationManager.waitTillReadyForBootstrap}} to error if a latch times out or
is interrupted.
> Bootstrap starts before migration responses have completed
> --
>
> Key: CASSANDRA-10731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10731
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> We have a number of failing tests running on slow VMs that are failing
> because {{MigrationManager.isReadyForBootstrap}} is return {{true}} when
> there are still inflight responses being processed to migration requests.
> This is happening because the {{MigrationTask.runMayThrow}} has completed but
> the {{IAsyncCallback.response}} is still running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10731) Bootstrap starts before migration responses have completed
[
https://issues.apache.org/jira/browse/CASSANDRA-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013672#comment-15013672
]
Mike Adamson commented on CASSANDRA-10731:
--
[~jjordan] With the above change in {{bootstrap_test.resumable_bootstrap_test}}
failed in the dtest run for it. I'm now questioning whether we should fail if a
latch wait times out. I think by doing that we are defeating the ability to
resume bootstrapping if the node we are bootstrapping from dies during the
process.
Let me know what you think and I'll take the change out again.
> Bootstrap starts before migration responses have completed
> --
>
> Key: CASSANDRA-10731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10731
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> We have a number of failing tests running on slow VMs that are failing
> because {{MigrationManager.isReadyForBootstrap}} is return {{true}} when
> there are still inflight responses being processed to migration requests.
> This is happening because the {{MigrationTask.runMayThrow}} has completed but
> the {{IAsyncCallback.response}} is still running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10731) Bootstrap starts before migration responses have completed
[
https://issues.apache.org/jira/browse/CASSANDRA-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013795#comment-15013795
]
Mike Adamson commented on CASSANDRA-10731:
--
If we are saying that a migration request failing or overrunning is an
indication of the schema not being in sync then we should keep the failure
mechanism. Maybe the test is not strictly correct in that case and is shutting
node1 down at the wrong point. I'll have a play with it and see if that is the
case.
> Bootstrap starts before migration responses have completed
> --
>
> Key: CASSANDRA-10731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10731
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> We have a number of failing tests running on slow VMs that are failing
> because {{MigrationManager.isReadyForBootstrap}} is return {{true}} when
> there are still inflight responses being processed to migration requests.
> This is happening because the {{MigrationTask.runMayThrow}} has completed but
> the {{IAsyncCallback.response}} is still running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10731) Bootstrap starts before migration responses have completed
[
https://issues.apache.org/jira/browse/CASSANDRA-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15024308#comment-15024308
]
Mike Adamson edited comment on CASSANDRA-10731 at 11/24/15 11:31 AM:
-
Sorry, my bad. I've changed it to poll off the queue now.
was (Author: mike_tr_adamson):
Sorry, my bad. I've changed it to poll of the queue now.
> Bootstrap starts before migration responses have completed
> --
>
> Key: CASSANDRA-10731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10731
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.0.x
>
>
> We have a number of failing tests running on slow VMs that are failing
> because {{MigrationManager.isReadyForBootstrap}} is return {{true}} when
> there are still inflight responses being processed to migration requests.
> This is happening because the {{MigrationTask.runMayThrow}} has completed but
> the {{IAsyncCallback.response}} is still running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11983) Migration task failed to complete
[ https://issues.apache.org/jira/browse/CASSANDRA-11983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322332#comment-15322332 ] Mike Adamson commented on CASSANDRA-11983: -- This is probably related to CASSANDRA-10731. This introduced a wait on the completion of in-flight migration tasks. This error is logged if the completion latch wait times out. The timeout is controlled by the cassandra.migration_task_wait_in_seconds system property and the default is 1. Could you try setting this system property to a higher value and see if that stops the error messages. > Migration task failed to complete > - > > Key: CASSANDRA-11983 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11983 > Project: Cassandra > Issue Type: Bug > Components: Lifecycle > Environment: Docker / Kubernetes running > Linux cassandra-21 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-1 (2016-03-06) > x86_64 GNU/Linux > openjdk version "1.8.0_91" > OpenJDK Runtime Environment (build 1.8.0_91-8u91-b14-1~bpo8+1-b14) > OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode) > Cassnadra 3.5 installed from > deb-src http://www.apache.org/dist/cassandra/debian 35x main >Reporter: Chris Love > > When nodes are boostrapping I am getting mulitple errors: "Migration task > failed to complete", from MigrationManager.java > The errors increase as more nodes are added to the ring, as I am creating a > ring of 1k nodes. > Cassandra yaml i here > https://github.com/k8s-for-greeks/gpmr/blob/3d50ff91a139b9c4a7a26eda0fb4dcf9a008fbed/pet-race-devops/docker/cassandra-debian/files/cassandra.yaml -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11022) Use SHA hashing to store password in the credentials cache
Mike Adamson created CASSANDRA-11022:
Summary: Use SHA hashing to store password in the credentials cache
Key: CASSANDRA-11022
URL: https://issues.apache.org/jira/browse/CASSANDRA-11022
Project: Cassandra
Issue Type: New Feature
Reporter: Mike Adamson
In CASSANDRA-7715 a credentials cache has been added to the
{{PasswordAuthenticator}} to improve performance when multiple authentications
occur for the same user.
Unfortunately, the bcrypt hash is being cached which is one of the major
performance overheads in password authentication.
I propose that the cache is changed to use a SHA- hash to store the user
password. As long as the cache is cleared for the user on an unsuccessful
authentication this won't significantly increase the ability of an attacker to
use a brute force attack because every other attempt will use bcrypt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7715) Add a credentials cache to the PasswordAuthenticator
[ https://issues.apache.org/jira/browse/CASSANDRA-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-7715: Reviewer: Mike Adamson > Add a credentials cache to the PasswordAuthenticator > > > Key: CASSANDRA-7715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7715 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Mike Adamson >Assignee: Sam Tunnicliffe >Priority: Minor > Fix For: 3.x > > > If the PasswordAuthenticator cached credentials for a short time it would > reduce the overhead of user journeys when they need to do multiple > authentications in quick succession. > This cache should work in the same way as the cache in CassandraAuthorizer in > that if it's TTL is set to 0 the cache will be disabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7715) Add a credentials cache to the PasswordAuthenticator
[
https://issues.apache.org/jira/browse/CASSANDRA-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15105461#comment-15105461
]
Mike Adamson commented on CASSANDRA-7715:
-
I'm not sure about being able to modify cache settings via JMX. This seems like
an attack vector to me. In a (very hypothetical) scenario, if an attacker got
hold of user credentials and JMX credentials, they could change the cache
timeouts so that a cache containing a known password would never change. I know
this is a fragile attack because all an administrator would need to do would be
to change the cache timeouts themselves and the cache would reinitialize.
I think that the cache should be be cleared for a user if the authentication
fails. Another small issue but it gives a small advantage to the attacker if
it's left in after a failure. This only really affects the {{CredentialsCache}}
in the {{PasswordAuthenticator}}.
Could we have a method to invalidate the cache for a specific user? I'm just
thinking that the only option an administrator has if they change a users
password is to {{invalidate}} the whole cache.
In cassandra.yaml the {{credentials_update_interval_in_ms}} value (although
commented out) ought to be the same value as {{credentials_validity_in_ms}}. We
are saying that it defaults to the same value so it probably ought to have it.
It would be nice if {{MBEAN_NAME_BASE}} was (somehow) overridable by concrete
implementations. This would make this class more usable by 3rd party
implementers.
> Add a credentials cache to the PasswordAuthenticator
>
>
> Key: CASSANDRA-7715
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7715
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Mike Adamson
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 3.x
>
>
> If the PasswordAuthenticator cached credentials for a short time it would
> reduce the overhead of user journeys when they need to do multiple
> authentications in quick succession.
> This cache should work in the same way as the cache in CassandraAuthorizer in
> that if it's TTL is set to 0 the cache will be disabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7715) Add a credentials cache to the PasswordAuthenticator
[ https://issues.apache.org/jira/browse/CASSANDRA-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144514#comment-15144514 ] Mike Adamson commented on CASSANDRA-7715: - +1 LGTM. I'm happy waiting for CASSANDRA-11022 clearing down on auth failures. As you say, it'll be far more relevant there. > Add a credentials cache to the PasswordAuthenticator > > > Key: CASSANDRA-7715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7715 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Mike Adamson >Assignee: Sam Tunnicliffe >Priority: Minor > Fix For: 3.x > > > If the PasswordAuthenticator cached credentials for a short time it would > reduce the overhead of user journeys when they need to do multiple > authentications in quick succession. > This cache should work in the same way as the cache in CassandraAuthorizer in > that if it's TTL is set to 0 the cache will be disabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11071) Invalidate legacy schema CFSs at startup
[
https://issues.apache.org/jira/browse/CASSANDRA-11071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson reassigned CASSANDRA-11071:
Assignee: Mike Adamson
> Invalidate legacy schema CFSs at startup
>
>
> Key: CASSANDRA-11071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11071
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata, Lifecycle
>Reporter: Sam Tunnicliffe
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.x, 3.x
>
>
> {{ColumnFamilyStore}} instances are created for legacy schema tables at
> startup when {{SystemKeyspace}} is initialized as they may be required for
> schema migration during upgrade. Before startup completes, the schema info
> for these is expunged from {{system_schema}}, but the {{CFS}} instances are
> not invalidated, which leaves their mbeans registered and visible via
> nodetool & JMX.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11022) Use SHA hashing to store password in the credentials cache
[
https://issues.apache.org/jira/browse/CASSANDRA-11022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-11022:
-
Fix Version/s: 3.4
> Use SHA hashing to store password in the credentials cache
> --
>
> Key: CASSANDRA-11022
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11022
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Mike Adamson
> Fix For: 3.4
>
>
> In CASSANDRA-7715 a credentials cache has been added to the
> {{PasswordAuthenticator}} to improve performance when multiple
> authentications occur for the same user.
> Unfortunately, the bcrypt hash is being cached which is one of the major
> performance overheads in password authentication.
> I propose that the cache is changed to use a SHA- hash to store the user
> password. As long as the cache is cleared for the user on an unsuccessful
> authentication this won't significantly increase the ability of an attacker
> to use a brute force attack because every other attempt will use bcrypt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11071) Invalidate legacy schema CFSs at startup
[
https://issues.apache.org/jira/browse/CASSANDRA-11071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119210#comment-15119210
]
Mike Adamson commented on CASSANDRA-11071:
--
[~iamaleksey] Yes, it fixes an inconsistency between the tables returned by
{{cqlsh> describe tables;}} and {{nodetool tablesstats}}.
> Invalidate legacy schema CFSs at startup
>
>
> Key: CASSANDRA-11071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11071
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata, Lifecycle
>Reporter: Sam Tunnicliffe
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.x, 3.x
>
>
> {{ColumnFamilyStore}} instances are created for legacy schema tables at
> startup when {{SystemKeyspace}} is initialized as they may be required for
> schema migration during upgrade. Before startup completes, the schema info
> for these is expunged from {{system_schema}}, but the {{CFS}} instances are
> not invalidated, which leaves their mbeans registered and visible via
> nodetool & JMX.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11071) Invalidate legacy schema CFSs at startup
[
https://issues.apache.org/jira/browse/CASSANDRA-11071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119172#comment-15119172
]
Mike Adamson commented on CASSANDRA-11071:
--
Here is a patch for this:
||cassandra-3.0||
|[branch|https://github.com/mike-tr-adamson/cassandra/tree/11071-cassandra-3.0]|
|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-11071-cassandra-3.0-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-11071-cassandra-3.0-dtest/]|
cc/ [~iamaleksey] was there a reason for not doing this?
> Invalidate legacy schema CFSs at startup
>
>
> Key: CASSANDRA-11071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11071
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata, Lifecycle
>Reporter: Sam Tunnicliffe
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.x, 3.x
>
>
> {{ColumnFamilyStore}} instances are created for legacy schema tables at
> startup when {{SystemKeyspace}} is initialized as they may be required for
> schema migration during upgrade. Before startup completes, the schema info
> for these is expunged from {{system_schema}}, but the {{CFS}} instances are
> not invalidated, which leaves their mbeans registered and visible via
> nodetool & JMX.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-11372) Make CQL grammar more easily extensible
Mike Adamson created CASSANDRA-11372:
Summary: Make CQL grammar more easily extensible
Key: CASSANDRA-11372
URL: https://issues.apache.org/jira/browse/CASSANDRA-11372
Project: Cassandra
Issue Type: Improvement
Components: CQL
Reporter: Mike Adamson
Assignee: Mike Adamson
Fix For: 3.5
The CQL grammar ({{Cql.g}}) is currently a composite grammar and, as such, is
not easy to extend.
We now have a number of 3rd parties who are extending the grammar (custom index
grammars, for example) and it would be helpful if the grammar could be split in
a parser and lexer in order to make extension easier.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11372) Make CQL grammar more easily extensible
[
https://issues.apache.org/jira/browse/CASSANDRA-11372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-11372:
-
Status: Patch Available (was: Open)
> Make CQL grammar more easily extensible
>
>
> Key: CASSANDRA-11372
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11372
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.x
>
>
> The CQL grammar ({{Cql.g}}) is currently a composite grammar and, as such, is
> not easy to extend.
> We now have a number of 3rd parties who are extending the grammar (custom
> index grammars, for example) and it would be helpful if the grammar could be
> split in a parser and lexer in order to make extension easier.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11372) Make CQL grammar more easily extensible
[
https://issues.apache.org/jira/browse/CASSANDRA-11372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199738#comment-15199738
]
Mike Adamson commented on CASSANDRA-11372:
--
Patch here:
||trunk||
|[branch|https://github.com/mike-tr-adamson/cassandra/tree/11372-trunk]|
|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-11372-trunk-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-11372-trunk-dtest/]|
> Make CQL grammar more easily extensible
>
>
> Key: CASSANDRA-11372
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11372
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.x
>
>
> The CQL grammar ({{Cql.g}}) is currently a composite grammar and, as such, is
> not easy to extend.
> We now have a number of 3rd parties who are extending the grammar (custom
> index grammars, for example) and it would be helpful if the grammar could be
> split in a parser and lexer in order to make extension easier.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11471) Add SASL mechanism negotiation to the native protocol
[
https://issues.apache.org/jira/browse/CASSANDRA-11471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15224121#comment-15224121
]
Mike Adamson commented on CASSANDRA-11471:
--
bq. The current AUTHENTICATE message sent from Client to Server includes the
java classname of the configured IAuthenticator. This could be superceded by a
new message which lists the SASL mechanisms supported by the server.
Why does this need to be separate message? Couldn't the list of supported
mechanisms be included with the {{AUTHENTICATE}} message?
> Add SASL mechanism negotiation to the native protocol
> -
>
> Key: CASSANDRA-11471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11471
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Sam Tunnicliffe
> Labels: client-impacting
>
> Introducing an additional message exchange into the authentication sequence
> would allow us to support multiple authentication schemes and [negotiation of
> SASL mechanisms|https://tools.ietf.org/html/rfc4422#section-3.2].
> The current {{AUTHENTICATE}} message sent from Client to Server includes the
> java classname of the configured {{IAuthenticator}}. This could be superceded
> by a new message which lists the SASL mechanisms supported by the server. The
> client would then respond with a new message which indicates it's choice of
> mechanism. This would allow the server to support multiple mechanisms, for
> example enabling both {{PLAIN}} for username/password authentication and
> {{EXTERNAL}} for a mechanism for extracting credentials from SSL
> certificates\* (see the example in
> [RFC-4422|https://tools.ietf.org/html/rfc4422#appendix-A]). Furthermore, the
> server could tailor the list of supported mechanisms on a per-connection
> basis, e.g. only offering certificate based auth to encrypted clients.
> The client's response should include the selected mechanism and any initial
> response data. This is mechanism-specific; the {{PLAIN}} mechanism consists
> of a single round in which the client sends encoded credentials as the
> initial response data and the server response indicates either success or
> failure with no futher challenges required.
> From a protocol perspective, after the mechanism negotiation the exchange
> would continue as in protocol v4, with one or more rounds of
> {{AUTH_CHALLENGE}} and {{AUTH_RESPONSE}} messages, terminated by an
> {{AUTH_SUCCESS}} sent from Server to Client upon successful authentication or
> an {{ERROR}} on auth failure.
> XMPP performs mechanism negotiation in this way,
> [RFC-3920|http://tools.ietf.org/html/rfc3920#section-6] includes a good
> overview.
> \* Note: this would require some a priori agreement between client and server
> over the implementation of the {{EXTERNAL}} mechanism.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12189) $$ escaped string literals are not handled correctly in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-12189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374740#comment-15374740
]
Mike Adamson commented on CASSANDRA-12189:
--
I have pushed a branch that fixes this to trunk but this is easily portable to
previous versions if needed.
||trunk||
|[branch|https://github.com/mike-tr-adamson/cassandra/tree/12189-trunk]|
|[testall|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-12189-trunk-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/madamson/job/mike-tr-adamson-12189-trunk-dtest/]|
> $$ escaped string literals are not handled correctly in cqlsh
> -
>
> Key: CASSANDRA-12189
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12189
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
> Fix For: 3.x
>
>
> The syntax rules for pg ($$) escaped string literals in cqlsh do not match
> the lexer rule for this type in Lexer.g.
> The {{unclosedPgString}} rule is not correctly matching pg string literals in
> multi-line statements so:
> {noformat}
> INSERT INTO test.test (id) values (
> ...$$
> {noformat}
> fails with a syntax error at the forward slash.
> Both {{pgStringLiteral}} and {{unclosedPgString}} fail with the following
> string
> {noformat}
> $$a$b$$
> {noformat}
> where this is allowed by the CQL lexer rule.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12189) $$ escaped string literals are not handled correctly in cqlsh
Mike Adamson created CASSANDRA-12189:
Summary: $$ escaped string literals are not handled correctly in
cqlsh
Key: CASSANDRA-12189
URL: https://issues.apache.org/jira/browse/CASSANDRA-12189
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Mike Adamson
Fix For: 3.x
The syntax rules for pg ($$) escaped string literals in cqlsh do not match the
lexer rule for this type in Lexer.g.
The {{unclosedPgString}} rule is not correctly matching pg string literals in
multi-line statements so:
{noformat}
INSERT INTO test.test (id) values (
...$$
{noformat}
fails with a syntax error at the forward slash.
Both {{pgStringLiteral}} and {{unclosedPgString}} fail with the following string
{noformat}
$$a$b$$
{noformat}
where this is allowed by the CQL lexer rule.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12189) $$ escaped string literals are not handled correctly in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-12189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382318#comment-15382318
]
Mike Adamson commented on CASSANDRA-12189:
--
bq. In context of CQLsh I don't see a way to distinguish between the end of
input and incorrectly closed $$. Do you think it's possible?
No, I don't. I couldn't find a way of covering all these cases so I opted for
letting it work correctly for valid input.
> $$ escaped string literals are not handled correctly in cqlsh
> -
>
> Key: CASSANDRA-12189
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12189
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.x
>
>
> The syntax rules for pg ($$) escaped string literals in cqlsh do not match
> the lexer rule for this type in Lexer.g.
> The {{unclosedPgString}} rule is not correctly matching pg string literals in
> multi-line statements so:
> {noformat}
> INSERT INTO test.test (id) values (
> ...$$
> {noformat}
> fails with a syntax error at the forward slash.
> Both {{pgStringLiteral}} and {{unclosedPgString}} fail with the following
> string
> {noformat}
> $$a$b$$
> {noformat}
> where this is allowed by the CQL lexer rule.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12187) $$ escaped string literals are not handled correctly in cqlsh
Mike Adamson created CASSANDRA-12187:
Summary: $$ escaped string literals are not handled correctly in
cqlsh
Key: CASSANDRA-12187
URL: https://issues.apache.org/jira/browse/CASSANDRA-12187
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Mike Adamson
Fix For: 3.x
The syntax rules for pg ($$) escaped string literals in cqlsh do not match the
lexer rule for this type in Lexer.g.
The {{unclosedPgString}} rule is not correctly matching pg string literals in
multi-line statements so:
{noformat}
INSERT INTO test.test (id) values (
...$$
{noformat}
fails with a syntax error at the forward slash.
Both {{pgStringLiteral}} and {{unclosedPgString}} fail with the following string
{noformat}
$$a$b$$
{noformat}
where this is allowed by the CQL lexer rule.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12188) $$ escaped string literals are not handled correctly in cqlsh
Mike Adamson created CASSANDRA-12188:
Summary: $$ escaped string literals are not handled correctly in
cqlsh
Key: CASSANDRA-12188
URL: https://issues.apache.org/jira/browse/CASSANDRA-12188
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Mike Adamson
Fix For: 3.x
The syntax rules for pg ($$) escaped string literals in cqlsh do not match the
lexer rule for this type in Lexer.g.
The {{unclosedPgString}} rule is not correctly matching pg string literals in
multi-line statements so:
{noformat}
INSERT INTO test.test (id) values (
...$$
{noformat}
fails with a syntax error at the forward slash.
Both {{pgStringLiteral}} and {{unclosedPgString}} fail with the following string
{noformat}
$$a$b$$
{noformat}
where this is allowed by the CQL lexer rule.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-12189) $$ escaped string literals are not handled correctly in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-12189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson reassigned CASSANDRA-12189:
Assignee: Mike Adamson
> $$ escaped string literals are not handled correctly in cqlsh
> -
>
> Key: CASSANDRA-12189
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12189
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.x
>
>
> The syntax rules for pg ($$) escaped string literals in cqlsh do not match
> the lexer rule for this type in Lexer.g.
> The {{unclosedPgString}} rule is not correctly matching pg string literals in
> multi-line statements so:
> {noformat}
> INSERT INTO test.test (id) values (
> ...$$
> {noformat}
> fails with a syntax error at the forward slash.
> Both {{pgStringLiteral}} and {{unclosedPgString}} fail with the following
> string
> {noformat}
> $$a$b$$
> {noformat}
> where this is allowed by the CQL lexer rule.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12189) $$ escaped string literals are not handled correctly in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-12189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-12189:
-
Status: Patch Available (was: Open)
> $$ escaped string literals are not handled correctly in cqlsh
> -
>
> Key: CASSANDRA-12189
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12189
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Assignee: Mike Adamson
> Fix For: 3.x
>
>
> The syntax rules for pg ($$) escaped string literals in cqlsh do not match
> the lexer rule for this type in Lexer.g.
> The {{unclosedPgString}} rule is not correctly matching pg string literals in
> multi-line statements so:
> {noformat}
> INSERT INTO test.test (id) values (
> ...$$
> {noformat}
> fails with a syntax error at the forward slash.
> Both {{pgStringLiteral}} and {{unclosedPgString}} fail with the following
> string
> {noformat}
> $$a$b$$
> {noformat}
> where this is allowed by the CQL lexer rule.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-16482) CQL schema generated by DESCRIBE KEYSPACE fails to import with custom indexes
Mike Adamson created CASSANDRA-16482:
Summary: CQL schema generated by DESCRIBE KEYSPACE fails to import
with custom indexes
Key: CASSANDRA-16482
URL: https://issues.apache.org/jira/browse/CASSANDRA-16482
Project: Cassandra
Issue Type: Bug
Components: CQL/Syntax
Reporter: Mike Adamson
The CQL schema generated by a DESCRIBE KEYSPACE on a keyspace with a table that
contains a custom index cannot be reimported.
The reimport of the schema fails with:
{noformat}
InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
specify class_name as a CUSTOM option"
{noformat}
The reason for this is that the custom index definition is created with
{{class_name}} and {{target}} options added to it in it's {{WITH OPTIONS}}
section. This is then picked up by the {{IndexAttributes.validate}} method that
specifically rejects them.
Note: This only seems to happen if the custom index was created with options in
the first place. So if the {{CREATE CUSTOM INDEX}} statement didn't originally
have a {{WITH OPTIONS}} section then it won't have the {{class_name}} and
{{target}} options added to it.
h3. Steps to reproduce
{noformat}
CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy',
'replication_factor': '1'};
CREATE TABLE test.test (id int PRIMARY KEY, value text);
CREATE CUSTOM INDEX ON test.test(value) USING
'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = {'is_literal':
'false'};
DESCRIBE KEYSPACE test;
{noformat}
Trying to import the resultant schema will fail.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18715) Add support for vector search in SAi
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson reassigned CASSANDRA-18715: Assignee: Mike Adamson > Add support for vector search in SAi > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAi
[
https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-18715:
-
Description: (was: Approximate nearest neighbour search results are
scored. As such, it is important that the results are returned in score order.
In order to do this proposal is that the ANN search is expressed in the grammar
as:
{code:java}
ORDER BY ANN OF {code}
rather than including the ANN search in the where clause.)
> Add support for vector search in SAi
>
>
> Key: CASSANDRA-18715
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18715
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Vector Search
>Reporter: Mike Adamson
>Priority: Normal
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18734) SAI result retriever is filtering too many rows
[ https://issues.apache.org/jira/browse/CASSANDRA-18734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18734: - Tester: Mike Adamson Status: Testing (was: Requires Testing) > SAI result retriever is filtering too many rows > --- > > Key: CASSANDRA-18734 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18734 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Attachments: flamegraph.svg, image-2023-08-08-13-48-19-701.png, > image-2023-08-08-13-49-19-576.png > > > Performance tests on SAI have shown that the number of rows being filtered > for wide row partitions is more than expected. For a 10k row/partition test - > limit 10, the following has been observed. > !image-2023-08-08-13-49-19-576.png|width=952,height=368! > This is not the expected outcome of row-awareness and needs investigating. > The number of rows read should, roughly, match the number of partitions read. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18734) SAI result retriever is filtering too many rows
[ https://issues.apache.org/jira/browse/CASSANDRA-18734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18734: - Status: Ready to Commit (was: Testing) > SAI result retriever is filtering too many rows > --- > > Key: CASSANDRA-18734 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18734 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Attachments: flamegraph.svg, image-2023-08-08-13-48-19-701.png, > image-2023-08-08-13-49-19-576.png > > > Performance tests on SAI have shown that the number of rows being filtered > for wide row partitions is more than expected. For a 10k row/partition test - > limit 10, the following has been observed. > !image-2023-08-08-13-49-19-576.png|width=952,height=368! > This is not the expected outcome of row-awareness and needs investigating. > The number of rows read should, roughly, match the number of partitions read. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18557) CEP-30 ANN Vector Search with SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson reassigned CASSANDRA-18557: Assignee: Mike Adamson > CEP-30 ANN Vector Search with SAI > - > > Key: CASSANDRA-18557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18557 > Project: Cassandra > Issue Type: Epic >Reporter: Jonathan Ellis >Assignee: Mike Adamson >Priority: Normal > > [CEP-30|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-30%3A+Approximate+Nearest+Neighbor%28ANN%29+Vector+Search+via+Storage-Attached+Indexes] > # Implement approximate nearest neighbor (ANN) vector search capability in > Apache Cassandra using storage-attached indexes (SAI). > # Support a vector of float32 embeddings as a new CQL type. > # Add ANN search to work with normal Cassandra data flow (insertion, > updating, and deleting rows). The implementation should support adding a new > vector in log(N) time, and ANN queries in M log(N) time where N is the number > of vectors and M is the number of sstables. > # Compose with other SAI predicates. > # Scatter/gather across replicas, combining topK from each to get global > topK. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18557) CEP-30 ANN Vector Search with SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18557: - Change Category: Operability Complexity: Byzantine Component/s: Feature/Vector Search Status: Open (was: Triage Needed) > CEP-30 ANN Vector Search with SAI > - > > Key: CASSANDRA-18557 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18557 > Project: Cassandra > Issue Type: Epic > Components: Feature/Vector Search >Reporter: Jonathan Ellis >Assignee: Mike Adamson >Priority: Normal > > [CEP-30|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-30%3A+Approximate+Nearest+Neighbor%28ANN%29+Vector+Search+via+Storage-Attached+Indexes] > # Implement approximate nearest neighbor (ANN) vector search capability in > Apache Cassandra using storage-attached indexes (SAI). > # Support a vector of float32 embeddings as a new CQL type. > # Add ANN search to work with normal Cassandra data flow (insertion, > updating, and deleting rows). The implementation should support adding a new > vector in log(N) time, and ANN queries in M log(N) time where N is the number > of vectors and M is the number of sstables. > # Compose with other SAI predicates. > # Scatter/gather across replicas, combining topK from each to get global > topK. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Test and Documentation Plan: Current test run: https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/288/workflows/e102cc8b-303a-400e-8265-f49cadf08eb5/jobs/19867 Status: Patch Available (was: In Progress) > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a CassandraOnHeapHnsw instance > * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk > components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk HNSW index -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Description: The patch associated with this ticket adds a new vector index to SAI. This introduces the following new elements and changes to SAI: * VectorMemtableIndex - the in-memory representation of the vector indexes that writes data to a CassandraOnHeapHnsw instance * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk components: ** VECTOR - contains the floating point vectors associated with the graph ** TERMS - contains the HNSW graph on-disk representation written by a HnswGraphWriter ** POSTINGS - contains the index postings as written by a VectorPostingsWriter * VectorIndexSegmentSearcher - used to search the on-disk HNSW index > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a CassandraOnHeapHnsw instance > * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk > components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk HNSW index -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17763728#comment-17763728 ] Mike Adamson commented on CASSANDRA-18715: -- |[5.0|https://github.com/apache/cassandra/pull/2673]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/288/workflows/e102cc8b-303a-400e-8265-f49cadf08eb5/jobs/19867]| This is an initial test run. I will continue to fix flakeys. > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18280) Investigate initial size of GrowableByteArrayDataOutput in RAMIndexOutput
[ https://issues.apache.org/jira/browse/CASSANDRA-18280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18280: - Resolution: Won't Fix Status: Resolved (was: Triage Needed) This has become obselete after the move to lucene 9.x > Investigate initial size of GrowableByteArrayDataOutput in RAMIndexOutput > - > > Key: CASSANDRA-18280 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18280 > Project: Cassandra > Issue Type: Task > Components: Feature/SAI >Reporter: Mike Adamson >Priority: Normal > > The GrowableByteArrayDataOutput in RAMIndexOutput is currently initialized > with a size of 128 bytes. There is no explanation as to why this size was > chosen. > The GrowableByteArrayDataOutput does not lazily allocate memory but only ever > allocates enough for each write operation. This can lead to a lot of fresh > allocations and calls to System.arrayCopy. > Since RAMIndexOutput is used to build the on-disk postings in SAI it is > likely that the size of the in-memory array is going to grow considerably > more that 128 bytes. > We should investigate changing this initial value to something higher and > possibly changing the GrowableByteArrayDataOutput class to allocate in blocks > rather than write increments. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18716) In-Memory write support for a vector search index
[ https://issues.apache.org/jira/browse/CASSANDRA-18716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18716: - Resolution: Duplicate Status: Resolved (was: Open) > In-Memory write support for a vector search index > - > > Key: CASSANDRA-18716 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18716 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > > This ticket will introduce the write side in-memory components of the vector > index. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAi
[
https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-18715:
-
Summary: Add support for vector search in SAi (was: Add support for ORDER
BY ANN OF )
> Add support for vector search in SAi
>
>
> Key: CASSANDRA-18715
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18715
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Vector Search
>Reporter: Mike Adamson
>Priority: Normal
>
> Approximate nearest neighbour search results are scored. As such, it is
> important that the results are returned in score order. In order to do this
> proposal is that the ANN search is expressed in the grammar as:
> {code:java}
> ORDER BY ANN OF {code}
> rather than including the ANN search in the where clause.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAi
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Change Category: Operability Complexity: Challenging Status: Open (was: Triage Needed) > Add support for vector search in SAi > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Summary: Add support for vector search in SAI (was: Add support for vector search in SAi) > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18734) SAI result retriever is filtering too many rows
[ https://issues.apache.org/jira/browse/CASSANDRA-18734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762448#comment-17762448 ] Mike Adamson commented on CASSANDRA-18734: -- Final test runs are here: |[JDK11|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/278/workflows/2b0d0a26-c175-4ef2-bf7d-6df982f9dfa6]| |[JDK17|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/278/workflows/1643c3f9-b55f-4285-bfce-0c792fc36a71]| There is a failure on the 17 test run that looks unrelated. > SAI result retriever is filtering too many rows > --- > > Key: CASSANDRA-18734 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18734 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Attachments: flamegraph.svg, image-2023-08-08-13-48-19-701.png, > image-2023-08-08-13-49-19-576.png > > > Performance tests on SAI have shown that the number of rows being filtered > for wide row partitions is more than expected. For a 10k row/partition test - > limit 10, the following has been observed. > !image-2023-08-08-13-49-19-576.png|width=952,height=368! > This is not the expected outcome of row-awareness and needs investigating. > The number of rows read should, roughly, match the number of partitions read. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779562#comment-17779562 ] Mike Adamson commented on CASSANDRA-18715: -- The latest CI runs are here: |[5.0|https://github.com/apache/cassandra/pull/2673]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/348/workflows/982fd591-c53a-4aa6-9a95-68f10df6bfae]| |[trunk|https://github.com/apache/cassandra/pull/2765]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/349/workflows/20dc9c0d-695a-4f85-b2e4-a568bac06bc6]| > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta, 5.x > > Time Spent: 24h 10m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18715: - Description: The patch associated with this ticket adds a new vector index to SAI. This introduces the following new elements and changes to SAI: * VectorMemtableIndex - the in-memory representation of the vector indexes that writes data to a DiskANN instance * VectorSegmentBuilder - that writes a DiskANN graph to the following on-disk components: ** VECTOR - contains the floating point vectors associated with the graph ** TERMS - contains the HNSW graph on-disk representation written by a HnswGraphWriter ** POSTINGS - contains the index postings as written by a VectorPostingsWriter * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph was: The patch associated with this ticket adds a new vector index to SAI. This introduces the following new elements and changes to SAI: * VectorMemtableIndex - the in-memory representation of the vector indexes that writes data to a CassandraOnHeapHnsw instance * VectorSegmentBuilder - that writes a HNSW graph to the following on-disk components: ** VECTOR - contains the floating point vectors associated with the graph ** TERMS - contains the HNSW graph on-disk representation written by a HnswGraphWriter ** POSTINGS - contains the index postings as written by a VectorPostingsWriter * VectorIndexSegmentSearcher - used to search the on-disk HNSW index > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta, 5.x > > Time Spent: 24h 10m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18994) SAI range query does not play together with "IN"
[
https://issues.apache.org/jira/browse/CASSANDRA-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson reassigned CASSANDRA-18994:
Assignee: Mike Adamson
> SAI range query does not play together with "IN"
>
>
> Key: CASSANDRA-18994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18994
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Stefan Miklosovic
>Assignee: Mike Adamson
>Priority: Normal
>
> I am using schema from the website's quickstart.
> {code}
> cqlsh> DESCRIBE KEYSPACE cycling ;
> CREATE KEYSPACE cycling WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE cycling.cyclist_semi_pro (
> id int PRIMARY KEY,
> affiliation text,
> age int,
> country text,
> firstname text,
> lastname text,
> registration date
> ) WITH additional_write_policy = '99p'
> AND allow_auto_snapshot = true
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND cdc = false
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '16', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND memtable = 'default'
> AND crc_check_chance = 1.0
> AND default_time_to_live = 0
> AND extensions = {}
> AND gc_grace_seconds = 864000
> AND incremental_backups = true
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair = 'BLOCKING'
> AND speculative_retry = '99p';
> CREATE CUSTOM INDEX age_sai_idx ON cycling.cyclist_semi_pro (age) USING
> 'StorageAttachedIndex';
> CREATE CUSTOM INDEX country_sai_idx ON cycling.cyclist_semi_pro (country)
> USING 'StorageAttachedIndex' WITH OPTIONS = {'ascii': 'true',
> 'case_sensitive': 'false', 'normalize': 'true'};
> CREATE CUSTOM INDEX lastname_sai_idx ON cycling.cyclist_semi_pro (lastname)
> USING 'StorageAttachedIndex' WITH OPTIONS = {'ascii': 'true',
> 'case_sensitive': 'false', 'normalize': 'true'};
> CREATE CUSTOM INDEX registration_sai_idx ON cycling.cyclist_semi_pro
> (registration) USING 'StorageAttachedIndex';
> {code}
> Then I do:
> {code}
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE lastname in ('Cantona',
> 'Boyd');
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> execute this query as it might involve data filtering and thus may have
> unpredictable performance. If you want to execute this query despite the
> performance unpredictability, use ALLOW FILTERING"
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE lastname in ('Cantona',
> 'Boyd') ALLOW FILTERING;
> id | affiliation | age | country | firstname | lastname | registration
> +-+-+-+---+--+--
> 5 | Como Velocità | 24 | ITA | Irene | Cantona | 2012-07-22
> 20 | London Cyclists | 18 | GBR |Leslie | Boyd | 2012-12-15
> {code}
> But check this:
> {code}
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE registration >
> '2010-01-01' AND registration < '2015-12-31' and lastname in ('Cantona',
> 'Boyd') allow filtering;
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read]
> message="Operation failed - received 0 responses and 1 failures: UNKNOWN from
> localhost/127.0.0.1:7000" info={'consistency': 'ONE', 'required_responses':
> 1, 'received_responses': 0, 'failures': 1, 'error_code_map': {'127.0.0.1':
> '0x'}}
> {code}
> and in the logs:
> {code}
> java.lang.AssertionError: null
> at
> org.apache.cassandra.index.sai.plan.Expression.add(Expression.java:171)
> at
> org.apache.cassandra.index.sai.plan.Operation.buildIndexExpressions(Operation.java:136)
> at
> org.apache.cassandra.index.sai.plan.Operation$AndNode.analyze(Operation.java:303)
> at
> org.apache.cassandra.index.sai.plan.Operation$Node.doTreeAnalysis(Operation.java:266)
> at
> org.apache.cassandra.index.sai.plan.Operation$Node.analyzeTree(Operation.java:251)
> at
> org.apache.cassandra.index.sai.plan.Operation.buildIterator(Operation.java:185)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.(StorageAttachedIndexSearcher.java:151)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher.search(StorageAttachedIndexSearcher.java:107)
> at
> org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:431)
> at
>
[jira] [Updated] (CASSANDRA-18994) SAI range query does not play together with "IN"
[
https://issues.apache.org/jira/browse/CASSANDRA-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-18994:
-
Bug Category: Parent values: Correctness(12982)
Complexity: Low Hanging Fruit
Discovered By: Adhoc Test
Severity: Low
Status: Open (was: Triage Needed)
> SAI range query does not play together with "IN"
>
>
> Key: CASSANDRA-18994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18994
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Stefan Miklosovic
>Assignee: Mike Adamson
>Priority: Normal
>
> I am using schema from the website's quickstart.
> {code}
> cqlsh> DESCRIBE KEYSPACE cycling ;
> CREATE KEYSPACE cycling WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE cycling.cyclist_semi_pro (
> id int PRIMARY KEY,
> affiliation text,
> age int,
> country text,
> firstname text,
> lastname text,
> registration date
> ) WITH additional_write_policy = '99p'
> AND allow_auto_snapshot = true
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND cdc = false
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '16', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND memtable = 'default'
> AND crc_check_chance = 1.0
> AND default_time_to_live = 0
> AND extensions = {}
> AND gc_grace_seconds = 864000
> AND incremental_backups = true
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair = 'BLOCKING'
> AND speculative_retry = '99p';
> CREATE CUSTOM INDEX age_sai_idx ON cycling.cyclist_semi_pro (age) USING
> 'StorageAttachedIndex';
> CREATE CUSTOM INDEX country_sai_idx ON cycling.cyclist_semi_pro (country)
> USING 'StorageAttachedIndex' WITH OPTIONS = {'ascii': 'true',
> 'case_sensitive': 'false', 'normalize': 'true'};
> CREATE CUSTOM INDEX lastname_sai_idx ON cycling.cyclist_semi_pro (lastname)
> USING 'StorageAttachedIndex' WITH OPTIONS = {'ascii': 'true',
> 'case_sensitive': 'false', 'normalize': 'true'};
> CREATE CUSTOM INDEX registration_sai_idx ON cycling.cyclist_semi_pro
> (registration) USING 'StorageAttachedIndex';
> {code}
> Then I do:
> {code}
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE lastname in ('Cantona',
> 'Boyd');
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> execute this query as it might involve data filtering and thus may have
> unpredictable performance. If you want to execute this query despite the
> performance unpredictability, use ALLOW FILTERING"
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE lastname in ('Cantona',
> 'Boyd') ALLOW FILTERING;
> id | affiliation | age | country | firstname | lastname | registration
> +-+-+-+---+--+--
> 5 | Como Velocità | 24 | ITA | Irene | Cantona | 2012-07-22
> 20 | London Cyclists | 18 | GBR |Leslie | Boyd | 2012-12-15
> {code}
> But check this:
> {code}
> cqlsh> SELECT * FROM cycling.cyclist_semi_pro WHERE registration >
> '2010-01-01' AND registration < '2015-12-31' and lastname in ('Cantona',
> 'Boyd') allow filtering;
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read]
> message="Operation failed - received 0 responses and 1 failures: UNKNOWN from
> localhost/127.0.0.1:7000" info={'consistency': 'ONE', 'required_responses':
> 1, 'received_responses': 0, 'failures': 1, 'error_code_map': {'127.0.0.1':
> '0x'}}
> {code}
> and in the logs:
> {code}
> java.lang.AssertionError: null
> at
> org.apache.cassandra.index.sai.plan.Expression.add(Expression.java:171)
> at
> org.apache.cassandra.index.sai.plan.Operation.buildIndexExpressions(Operation.java:136)
> at
> org.apache.cassandra.index.sai.plan.Operation$AndNode.analyze(Operation.java:303)
> at
> org.apache.cassandra.index.sai.plan.Operation$Node.doTreeAnalysis(Operation.java:266)
> at
> org.apache.cassandra.index.sai.plan.Operation$Node.analyzeTree(Operation.java:251)
> at
> org.apache.cassandra.index.sai.plan.Operation.buildIterator(Operation.java:185)
> at
> org.apache.cassandra.index.sai.plan.StorageAttachedIndexSearcher$ResultRetriever.(StorageAttachedIndexSearcher.java:151)
> at
>
[jira] [Updated] (CASSANDRA-18166) Improve the code model around IndexContext
[ https://issues.apache.org/jira/browse/CASSANDRA-18166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18166: - Change Category: Operability Complexity: Normal Status: Open (was: Triage Needed) > Improve the code model around IndexContext > -- > > Key: CASSANDRA-18166 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18166 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Labels: SAI > > We currently have a situation where we need to create an IndexContext that is > for a non-indexed column and therefore is never going to be used for indexing > or searching. This results in the IndexContext having to check for this at > points in the code with assertions. The reason for this that, even when the > column is non-indexed, we need to have information about the column for the > purpose of post-filtering. > It would make sense to split out the column / index information needed for > filtering from the indexing / searching requirements such that we could avoid > unnecessary assertions in the code. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781092#comment-17781092 ] Mike Adamson commented on CASSANDRA-18715: -- [~adelapena] Yes, I would prefer that. I am actively working on the distributed test failures. I will have a look at the trunk failures as well. > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Attachments: signature.asc > > Time Spent: 25h 40m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780972#comment-17780972 ] Mike Adamson commented on CASSANDRA-18715: -- PRs and CI for fixing VectorUpdateDeleteTest are here: |[5.0|https://github.com/apache/cassandra/pull/2850]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/357/workflows/46d6ab84-a2e9-4bcd-8a08-3ae2089476c9]| > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Time Spent: 25h 40m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780972#comment-17780972 ] Mike Adamson edited comment on CASSANDRA-18715 at 10/30/23 11:42 AM: - PRs and CI for fixing VectorUpdateDeleteTest are here: |[5.0|https://github.com/apache/cassandra/pull/2850]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/357/workflows/46d6ab84-a2e9-4bcd-8a08-3ae2089476c9]| |[trunk|https://github.com/apache/cassandra/pull/2851]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/359/workflows/af539922-7531-447b-ae41-dd63cf0fe627]| was (Author: mike_tr_adamson): PRs and CI for fixing VectorUpdateDeleteTest are here: |[5.0|https://github.com/apache/cassandra/pull/2850]|[CI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/357/workflows/46d6ab84-a2e9-4bcd-8a08-3ae2089476c9]| > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Time Spent: 25h 40m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780980#comment-17780980 ] Mike Adamson commented on CASSANDRA-18715: -- [~smiklosovic] I would prefer to merge this one because it is a failure for every run and put a separate patch for the rangeRestrictedTest. That is a random test failure and is going to take a little longer to fix. I am actively working on it, so hopefully not too long. > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Time Spent: 25h 40m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780932#comment-17780932 ] Mike Adamson commented on CASSANDRA-18715: -- [~smiklosovic] [~mck] I am testing a patch for this. I will post the patch here with test runs when the test runs are complete. > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Time Spent: 25.5h > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18715) Add support for vector search in SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779830#comment-17779830 ] Mike Adamson commented on CASSANDRA-18715: -- The contributors of the patch associated with this ticket wish to make it clear that the contribution adheres to clauses #5 & #7 of the Apache Foundation [ICLA|https://www.apache.org/licenses/icla.pdf]. > Add support for vector search in SAI > > > Key: CASSANDRA-18715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18715 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Vector Search >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta, 5.x > > Time Spent: 24h 20m > Remaining Estimate: 0h > > The patch associated with this ticket adds a new vector index to SAI. This > introduces the following new elements and changes to SAI: > * VectorMemtableIndex - the in-memory representation of the vector indexes > that writes data to a DiskANN instance > * VectorSegmentBuilder - that writes a DiskANN graph to the following > on-disk components: > ** VECTOR - contains the floating point vectors associated with the graph > ** TERMS - contains the HNSW graph on-disk representation written by a > HnswGraphWriter > ** POSTINGS - contains the index postings as written by a > VectorPostingsWriter > * VectorIndexSegmentSearcher - used to search the on-disk DiskANN graph -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
