>From Ian Maxon <[email protected]>: Attention is currently required from: Hongyu Shi.
Ian Maxon has uploaded a new patch set (#17) to the change originally created by Hongyu Shi. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21287?usp=email ) The following approvals got outdated and were removed: Contrib-2 by Anon. E. Moose #1000171, Integration-Tests-1 by Jenkins, Verified-1 by Jenkins Change subject: [ASTERIXDB-3771] ANN query optimizer rule for top-k ANN vector search ...................................................................... [ASTERIXDB-3771] ANN query optimizer rule for top-k ANN vector search - user model changes: yes - storage format changes: no - interface changes: yes Add optimizer rules and physical operators for approximate nearest neighbor (ANN) top-k queries using VTree vector indexes. Components: - IntroduceTopKAccessMethodRule: matches LIMIT k -> ORDER BY ann_distance(...) pattern and transforms to vector index access - VectorIndexAccessMethod: implements IAccessMethod for vector index selection - VectorSearchPOperator: physical operator for vector search execution - PushFilterIntoVectorSearchRule: pushes INCLUDE field filters into vector search - ANNDistanceDescriptor: runtime evaluator for ann_distance function - VectorDistanceFunctionFactory: factory for distance metric implementations - Metadata/DDL integration for vector index query path Change-Id: Iaa502b57915c75921cd5aa5c62fc0e24261bb0f6 --- A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/VectorIndexFilterSchema.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/VectorIndexFilterTypeEnvironment.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/VectorSearchPOperator.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/SqlppCompilationProvider.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFilterIntoVectorSearchRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetAsterixPhysicalOperatorsRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceTopKAccessMethodRule.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/VectorIndexAccessMethod.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/VectorJobGenParams.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/ValidateUtil.java A asterixdb/asterix-app/data/vector/glove-sub.adm A asterixdb/asterix-app/data/vector/movie-sub.adm M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DatasetRewriter.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/io/PersistedResourceRegistry.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java M asterixdb/asterix-app/src/test/resources/cc-main.conf M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/VectorQueries.xml A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.3.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.4.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.5.plans.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.99.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-glove/create-index-vtree-glove.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-glove/create-index-vtree-glove.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-glove/create-index-vtree-glove.3.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-glove/create-index-vtree-glove.4.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-glove/create-index-vtree-glove.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-glove/create-index-vtree-glove.99.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-movie/create-index-vtree-movie.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-movie/create-index-vtree-movie.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-movie/create-index-vtree-movie.3.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-movie/create-index-vtree-movie.4.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-movie/create-index-vtree-movie.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-movie/create-index-vtree-movie.99.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree/create-index-vtree.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree/create-index-vtree.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree/create-index-vtree.3.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree/create-index-vtree.4.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree/create-index-vtree.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree/create-index-vtree.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree/create-index-vtree.7.plans.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree/create-index-vtree.99.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.5.regex A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.6.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-glove/create-index-vtree-glove.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-movie/create-index-vtree-movie.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree/create-index-vtree.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree/create-index-vtree.6.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree/create-index-vtree.7.regex M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/DatasetLocalResource.java M asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/statement/CreateIndexStatement.java M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/CommonFunctionMapUtil.java M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/FunctionUtil.java A asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/VectorIndexDeclUtil.java M asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj M asterixdb/asterix-metadata/pom.xml M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/BTreeResourceFactoryProvider.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataManagerUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataProvider.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Dataset.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/IndexTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/DatasetUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/KeyFieldTypeUtil.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/vector/ANNDistanceDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/functions/FunctionCollection.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/PhysicalOperatorTag.java 75 files changed, 5,565 insertions(+), 58 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/87/21287/17 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21287?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: newpatchset Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Iaa502b57915c75921cd5aa5c62fc0e24261bb0f6 Gerrit-Change-Number: 21287 Gerrit-PatchSet: 17 Gerrit-Owner: Hongyu Shi <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Attention: Hongyu Shi <[email protected]>
