>From Ali Alsuliman <[email protected]>: Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21623?usp=email )
Change subject: [ASTERIXDB-3817][COMP] Make TYPE VTREE optional for VECTOR fields ...................................................................... [ASTERIXDB-3817][COMP] Make TYPE VTREE optional for VECTOR fields A VECTOR field annotation required an explicit TYPE VTREE. No other index type accepts a VECTOR field, so the annotation alone determines the index type; TYPE VTREE is now optional and inferred from it. An explicit type that is not VTREE still fails, and so does TYPE VTREE without the annotation -- nothing else identifies the vector field. The annotation was also detected only on the first project of a lone indexed element, so a VECTOR appearing anywhere else fell through to a regular index, where the internal marker was resolved as a type name: CREATE INDEX i ON ds(embedding VECTOR, year) ASX1082: Cannot find datatype with name $$VECTOR_MARKER$$ Detection now scans every element and every project, so that case reports the one-field rule the code already carried but could not reach. It also fixes the explicit-TYPE-VTREE spelling of the same mistake, which used to claim the VECTOR annotation was missing when it was not. Making TYPE optional is what turns this from a corner into a mistake a user can easily make. Ext-ref: MB-73194 Co-Authored-By: Claude Opus 5 <[email protected]> Change-Id: I913ef001af90a23bdbdd04e9d85009d83c36a501 --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/VectorQueries.xml A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-multi-field/create-index-vtree-multi-field.01.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.01.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.02.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.03.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.04.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.05.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.06.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.99.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-type-mismatch/create-index-vtree-type-mismatch.01.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-no-type/create-index-vtree-no-type.05.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-no-type/create-index-vtree-no-type.06.adm M asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj 13 files changed, 313 insertions(+), 19 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/23/21623/1 diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/VectorQueries.xml b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/VectorQueries.xml index 4955fef..99bd8d7 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/VectorQueries.xml +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/VectorQueries.xml @@ -75,6 +75,23 @@ </compilation-unit> </test-case> <test-case FilePath="vector"> + <compilation-unit name="create-index-vtree-no-type"> + <output-dir compare="Clean-JSON">create-index-vtree-no-type</output-dir> + </compilation-unit> + </test-case> + <test-case FilePath="vector"> + <compilation-unit name="create-index-vtree-type-mismatch"> + <output-dir compare="Text">create-index-vtree-type-mismatch</output-dir> + <expected-error>VECTOR field annotation is not supported for TYPE BTREE</expected-error> + </compilation-unit> + </test-case> + <test-case FilePath="vector"> + <compilation-unit name="create-index-vtree-multi-field"> + <output-dir compare="Text">create-index-vtree-multi-field</output-dir> + <expected-error>Vector index requires exactly one field</expected-error> + </compilation-unit> + </test-case> + <test-case FilePath="vector"> <!-- Either build job can report a wholly non-indexable sample: the quantization job names the declared and the found dimension, and the training job names the declared one among the causes of an empty parameter block. Which one arrives first is not deterministic, so match the phrase diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-multi-field/create-index-vtree-multi-field.01.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-multi-field/create-index-vtree-multi-field.01.ddl.sqlpp new file mode 100644 index 0000000..4aff049 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-multi-field/create-index-vtree-multi-field.01.ddl.sqlpp @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +// A VECTOR field can only be indexed on its own. With TYPE optional this is easy to reach by +// accident, so it must report the one-field rule rather than the internal $$VECTOR_MARKER$$ name. +DROP DATAVERSE test IF EXISTS; +CREATE DATAVERSE test; +USE test; +CREATE TYPE MovieType AS { id: int, year: int, embedding: [double] }; +CREATE DATASET MovieSmall(MovieType) PRIMARY KEY id; +CREATE INDEX idx_multi ON MovieSmall(embedding VECTOR, year); diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.01.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.01.ddl.sqlpp new file mode 100644 index 0000000..2497935 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.01.ddl.sqlpp @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +// TYPE VTREE is optional when the indexed field carries the VECTOR annotation: no other index type +// accepts one, so the annotation alone determines the type. Everything below is deliberately the +// same shape as create-index-vtree, minus the TYPE clause. +DROP DATAVERSE test IF EXISTS; +CREATE DATAVERSE test; +USE test; +CREATE TYPE MovieType AS { + id: int, + embedding: [double] +}; +CREATE DATASET MovieSmall(MovieType) PRIMARY KEY id; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.02.update.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.02.update.sqlpp new file mode 100644 index 0000000..94517b1 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.02.update.sqlpp @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +USE test; + +// The 4th component is the only thing that varies within each block, so the 3 nearest to the +// origin are ids 1, 2 and 3 regardless of how the two blocks are clustered. +INSERT INTO MovieSmall ([ + {"id": 1, "embedding": [0.0, 0.0, 0.0, 0.1]}, + {"id": 2, "embedding": [0.0, 0.0, 0.0, 0.2]}, + {"id": 3, "embedding": [0.0, 0.0, 0.0, 0.3]}, + {"id": 4, "embedding": [0.0, 0.0, 0.0, 0.4]}, + {"id": 5, "embedding": [0.0, 0.0, 0.0, 0.5]}, + {"id": 6, "embedding": [0.0, 0.0, 0.0, 0.6]}, + {"id": 7, "embedding": [0.0, 0.0, 0.0, 0.7]}, + {"id": 8, "embedding": [0.0, 0.0, 0.0, 0.8]}, + {"id": 9, "embedding": [0.0, 0.0, 0.0, 0.9]}, + {"id": 10, "embedding": [0.0, 0.0, 0.0, 1.0]}, + {"id": 11, "embedding": [0.0, 0.0, 0.0, 1.1]}, + {"id": 12, "embedding": [0.0, 0.0, 0.0, 1.2]}, + {"id": 13, "embedding": [0.0, 0.0, 0.0, 1.3]}, + {"id": 14, "embedding": [0.0, 0.0, 0.0, 1.4]}, + {"id": 15, "embedding": [0.0, 0.0, 0.0, 1.5]}, + {"id": 16, "embedding": [10.0, 10.0, 10.0, 0.1]}, + {"id": 17, "embedding": [10.0, 10.0, 10.0, 0.2]}, + {"id": 18, "embedding": [10.0, 10.0, 10.0, 0.3]}, + {"id": 19, "embedding": [10.0, 10.0, 10.0, 0.4]}, + {"id": 20, "embedding": [10.0, 10.0, 10.0, 0.5]}, + {"id": 21, "embedding": [10.0, 10.0, 10.0, 0.6]}, + {"id": 22, "embedding": [10.0, 10.0, 10.0, 0.7]}, + {"id": 23, "embedding": [10.0, 10.0, 10.0, 0.8]}, + {"id": 24, "embedding": [10.0, 10.0, 10.0, 0.9]}, + {"id": 25, "embedding": [10.0, 10.0, 10.0, 1.0]}, + {"id": 26, "embedding": [10.0, 10.0, 10.0, 1.1]}, + {"id": 27, "embedding": [10.0, 10.0, 10.0, 1.2]}, + {"id": 28, "embedding": [10.0, 10.0, 10.0, 1.3]}, + {"id": 29, "embedding": [10.0, 10.0, 10.0, 1.4]}, + {"id": 30, "embedding": [10.0, 10.0, 10.0, 1.5]} +]); diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.03.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.03.ddl.sqlpp new file mode 100644 index 0000000..b63fc7a --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.03.ddl.sqlpp @@ -0,0 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +// A sample index is required before creating a vector index (quantization training scans it). +ANALYZE DATASET test.MovieSmall WITH {"sample-seed": 1000}; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.04.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.04.ddl.sqlpp new file mode 100644 index 0000000..62d88b6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.04.ddl.sqlpp @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +USE test; + +// No TYPE clause. The VECTOR annotation on `embedding` is what makes this a vector index; the +// INCLUDE/WITH/EXCLUDE UNKNOWN KEY clauses that only a vector index accepts must be honoured too. +CREATE INDEX idx_emb + ON MovieSmall(embedding VECTOR) + WITH { "dimension": 4, "similarity": "euclidean", "num_clusters": 2, "train_list_fraction": 1.0, "seed": 42 } + EXCLUDE UNKNOWN KEY; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.05.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.05.query.sqlpp new file mode 100644 index 0000000..394836f --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.05.query.sqlpp @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +// The persisted index structure is VTREE even though the DDL never said so -- an inferred type that +// came out as BTREE would be caught here rather than surfacing as a wrong plan much later. +SELECT i.IndexStructure, i.SearchKey +FROM Metadata.`Index` i +WHERE i.DataverseName = "test" AND i.DatasetName = "MovieSmall" AND i.IndexName = "idx_emb"; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.06.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.06.query.sqlpp new file mode 100644 index 0000000..dad4664 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.06.query.sqlpp @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +USE test; + +// The inferred index is also usable: min_probe_fraction 1.0 scans every cluster, so the result is +// clustering-independent. The 3 nearest to the origin are the smallest near-origin embeddings. +SELECT VALUE m.id +FROM MovieSmall m +ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0) +LIMIT 3; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.99.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.99.ddl.sqlpp new file mode 100644 index 0000000..d240ef2 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-no-type/create-index-vtree-no-type.99.ddl.sqlpp @@ -0,0 +1,19 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +DROP DATAVERSE test IF EXISTS; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-type-mismatch/create-index-vtree-type-mismatch.01.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-type-mismatch/create-index-vtree-type-mismatch.01.ddl.sqlpp new file mode 100644 index 0000000..2e14d91 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-type-mismatch/create-index-vtree-type-mismatch.01.ddl.sqlpp @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +// TYPE is optional for a VECTOR field, but an explicit type that is not VTREE contradicts the +// annotation and must be rejected rather than quietly building a BTREE over the vector field. +DROP DATAVERSE test IF EXISTS; +CREATE DATAVERSE test; +USE test; + +CREATE TYPE MovieType AS { + id: int, + embedding: [double] +}; + +CREATE DATASET MovieSmall(MovieType) PRIMARY KEY id; + +CREATE INDEX idx_emb + ON MovieSmall(embedding VECTOR) + TYPE BTREE + WITH { "dimension": 4, "similarity": "euclidean" } + EXCLUDE UNKNOWN KEY; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-no-type/create-index-vtree-no-type.05.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-no-type/create-index-vtree-no-type.05.adm new file mode 100644 index 0000000..1f82492 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-no-type/create-index-vtree-no-type.05.adm @@ -0,0 +1 @@ +{ "IndexStructure": "VTREE", "SearchKey": [ [ "embedding" ] ] } diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-no-type/create-index-vtree-no-type.06.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-no-type/create-index-vtree-no-type.06.adm new file mode 100644 index 0000000..01e79c3 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-no-type/create-index-vtree-no-type.06.adm @@ -0,0 +1,3 @@ +1 +2 +3 diff --git a/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj b/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj index 9f46c36..135043d 100644 --- a/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj +++ b/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj @@ -1704,41 +1704,55 @@ )? ) { - // Check if VECTOR field type annotation is present + // Check if a VECTOR field type annotation is present. Every element and every project is scanned, + // not just the first project of a lone element: a VECTOR that appears anywhere else must still + // reach the vector-index diagnostics below. Detecting only the single-element case let + // `(embedding VECTOR, year)` fall through to a regular index, where the marker was resolved as a + // type name and surfaced as "Cannot find datatype with name $$VECTOR_MARKER$$". boolean hasVectorField = false; - if (indexedElementList.size() == 1) { - CreateIndexStatement.IndexedElement field = indexedElementList.get(0); + for (CreateIndexStatement.IndexedElement field : indexedElementList) { List<Pair<List<String>, IndexedTypeExpression>> projectList = field.getProjectList(); - if (projectList != null && !projectList.isEmpty()) { - IndexedTypeExpression typeExpr = projectList.get(0).second; - if (typeExpr != null) { - TypeExpression innerType = typeExpr.getType(); - if (innerType != null && innerType instanceof TypeReferenceExpression) { - TypeReferenceExpression typeRef = (TypeReferenceExpression) innerType; - Pair<Namespace, Identifier> ident = typeRef.getIdent(); - if (ident != null && ident.second != null && - "$$VECTOR_MARKER$$".equals(ident.second.getValue())) { - hasVectorField = true; - } + if (projectList == null) { + continue; + } + for (Pair<List<String>, IndexedTypeExpression> project : projectList) { + IndexedTypeExpression typeExpr = project.second; + if (typeExpr == null) { + continue; + } + TypeExpression innerType = typeExpr.getType(); + if (innerType instanceof TypeReferenceExpression) { + Pair<Namespace, Identifier> ident = ((TypeReferenceExpression) innerType).getIdent(); + if (ident != null && ident.second != null && + "$$VECTOR_MARKER$$".equals(ident.second.getValue())) { + hasVectorField = true; } } } } - boolean isVTree = indexParams != null && indexParams.type == IndexType.VTREE; + boolean explicitVTree = indexParams != null && indexParams.type == IndexType.VTREE; - // Validate: VECTOR annotation requires TYPE VTREE - if (hasVectorField && !isVTree) { + // Validate: a VECTOR field is indexable only by a vector index, so an explicit type other than + // VTREE contradicts the annotation. TYPE may be omitted entirely -- see below. + if (hasVectorField && indexParams != null && !explicitVTree) { throw new SqlppParseException(getSourceLocation(startStmtToken), - "VECTOR field annotation requires TYPE VTREE. Expected syntax: CREATE INDEX ... (field VECTOR) TYPE VTREE WITH {...} EXCLUDE UNKNOWN KEY"); + "VECTOR field annotation is not supported for TYPE " + indexParams.type + + ". Expected syntax: CREATE INDEX ... (field VECTOR) [TYPE VTREE] WITH {...} EXCLUDE UNKNOWN KEY"); } // Validate: TYPE VTREE requires VECTOR annotation - if (isVTree && !hasVectorField) { + if (explicitVTree && !hasVectorField) { throw new SqlppParseException(getSourceLocation(startStmtToken), "TYPE VTREE requires VECTOR field annotation. Expected syntax: CREATE INDEX ... (field VECTOR) TYPE VTREE WITH {...} EXCLUDE UNKNOWN KEY"); } + // TYPE VTREE is optional: no other index type accepts a VECTOR field, so the annotation alone + // determines the index type. The two checks above have already rejected every case where the + // annotation and an explicit type disagree, which is what leaves this equivalence exact -- a + // VECTOR field with no TYPE clause would otherwise fall through to the BTREE/ARRAY default. + boolean isVTree = hasVectorField; + if (isVTree) { // Validate exactly one field for vector index if (indexedElementList.size() != 1) { -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21623?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I913ef001af90a23bdbdd04e9d85009d83c36a501 Gerrit-Change-Number: 21623 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman <[email protected]>
