>From Ali Alsuliman <[email protected]>: Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21682?usp=email )
Change subject: [ASTERIXDB-3817][STO] Take index-fixed search values from the index ...................................................................... [ASTERIXDB-3817][STO] Take index-fixed search values from the index The search operator shipped the vector accessor factory and the distance-function factory to the storage layer through the IIndexAccessParameters map. Neither carries information. Both are built from the index's own persisted parameters -- MetadataProvider reads vectorParameters.getSimilarity() for the metric and constructs a zero-arg AOrderedListVectorBinaryAccessorFactory whose body says "No configuration needed for now" -- and VTreeResourceFactoryProvider builds equivalent instances from the same values for the index itself. LSMVTree then hands its own copies to each memory component's VTree, so the map was carrying a third. There is no query override to preserve. The metric is chosen at CREATE INDEX time, and IntroduceTopKAccessMethodRule falls back to a full scan rather than using the index when a query asks for a different one, so a per-query metric can never reach the storage layer. resolveDistanceFunctionFactory's fallback to the index's own factory was therefore the only branch that could ever be taken; it is now a field read. Both values come from the index now: VTree from its own fields, and the top-K cursor from LSMVTree, which already held one. That removes the two entries, the two constructor arguments that fed them, and the failure mode of a search that cannot proceed because a parameter it never needed was not set. What is left in the map is the task context, which is genuinely per-operation and is how the inverted index passes its own; the quantizer factory, which is index-fixed too but cannot be taken from the index because OptimizedScalarQuantizerFactory lives in asterix-common and routing it through the resource means persisting it on the JSON-serialized LSMVTreeLocalResource; and a pre-built quantizer used as a test seam. VTreeSearchParameters documents that and reads what remains by type, separating "not set" from "set to the wrong type", which IIndexAccessParameters.getParameter cannot do because it answers null to both. Also fixed, found while testing the diagnostics: doDestroy called CleanupUtils.destroy on the per-component arrays without checking they exist. doOpen validates before allocating them, so a cursor whose open failed reached destroy() with nothing to reclaim and threw a NullPointerException from the cleanup path -- replacing whatever diagnosis doOpen had produced. Any open-time failure in that cursor was being masked, not only the ones this change added. Ext-ref: MB-73194 Co-Authored-By: Claude Opus 5 <[email protected]> Change-Id: I41fdd9e8c380b93d45c2626f14079c32a9f0d060 --- M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/dataflow/VTreeSearchOperatorNodePushable.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/impls/LSMVTree.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/impls/LSMVTreeTopKSearchCursor.java M hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTree.java A hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/utils/VTreeSearchParameters.java M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-vtree-test/src/test/java/org/apache/hyracks/storage/am/lsm/vector/LSMVTreeCursorAgreementTest.java 6 files changed, 226 insertions(+), 31 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/82/21682/1 diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/dataflow/VTreeSearchOperatorNodePushable.java b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/dataflow/VTreeSearchOperatorNodePushable.java index 9740548..91571e9 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/dataflow/VTreeSearchOperatorNodePushable.java +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/dataflow/VTreeSearchOperatorNodePushable.java @@ -343,15 +343,12 @@ @Override protected void addAdditionalIndexAccessorParams(IIndexAccessParameters iap) { - // Vector accessor factory: storage layer uses this to extract the query vector from the - // search predicate's tuple, keeping the extraction in the storage layer (no AsterixDB types - // leak down). - iap.getParameters().put(IVTreeBinaryAccessorFactory.IAP_KEY, vectorAccessorFactory); - - // Distance-function factory injected from AsterixDB so the VTree can build an - // IVTreeDistanceFunction without depending on asterix-runtime types. - iap.getParameters().put(IVTreeDistanceFunctionFactory.IAP_KEY, distanceFunctionFactory); - + // The vector accessor factory and the distance-function factory are deliberately NOT passed here. + // Both are fixed by the index -- the metric is chosen at CREATE INDEX time, and the optimizer falls + // back to a full scan rather than using the index when a query asks for a different one -- and the + // index already holds instances built from the same metadata this operator reads. Passing them + // meant every search carried two values that could only ever equal what the storage layer had. + // // Quantizer factory (nullable). The VTree builds a per-query IVTreeQuantizer from the // float[6] params persisted on the index. Null for non-quantized indexes and for test // contexts that inject a pre-built IVTreeQuantizer under IVTreeQuantizer.IAP_KEY. diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/impls/LSMVTree.java b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/impls/LSMVTree.java index f35746f..cc971c3 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/impls/LSMVTree.java +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/impls/LSMVTree.java @@ -648,6 +648,15 @@ * search cursor for the VTree index; the streaming {@link LSMVTreeSearchCursor} is reserved * for component merges. */ + /** + * The accessor that decodes a query vector. Fixed by the index's field type, so it is taken from here + * rather than passed per search: the search operator used to put an equivalent instance into the + * access-parameters map, which the index already held. + */ + public IVTreeBinaryAccessorFactory getVectorAccessorFactory() { + return vectorAccessorFactory; + } + public IIndexCursor createTopKSearchCursor(ILSMIndexOperationContext opCtx) { return new LSMVTreeTopKSearchCursor(opCtx); } diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/impls/LSMVTreeTopKSearchCursor.java b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/impls/LSMVTreeTopKSearchCursor.java index c9b442b..0b4c391 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/impls/LSMVTreeTopKSearchCursor.java +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-vtree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/impls/LSMVTreeTopKSearchCursor.java @@ -26,11 +26,9 @@ import java.util.PriorityQueue; import java.util.Set; -import org.apache.hyracks.api.context.IHyracksTaskContext; import org.apache.hyracks.api.exceptions.ErrorCode; import org.apache.hyracks.api.exceptions.HyracksDataException; import org.apache.hyracks.api.util.CleanupUtils; -import org.apache.hyracks.api.util.HyracksConstants; import org.apache.hyracks.dataflow.common.data.accessors.ITupleReference; import org.apache.hyracks.dataflow.common.utils.TupleUtils; import org.apache.hyracks.storage.am.common.api.ITupleFilter; @@ -41,7 +39,6 @@ import org.apache.hyracks.storage.am.lsm.vector.utils.LSMVTreeUtils; import org.apache.hyracks.storage.am.lsm.vector.utils.VTreeMergeKey; import org.apache.hyracks.storage.am.vector.api.IVTreeBinaryAccessor; -import org.apache.hyracks.storage.am.vector.api.IVTreeBinaryAccessorFactory; import org.apache.hyracks.storage.am.vector.api.IVTreeDistanceFunction; import org.apache.hyracks.storage.am.vector.api.IVTreeQuantizer; import org.apache.hyracks.storage.am.vector.impls.ClusterSearchResult; @@ -50,6 +47,7 @@ import org.apache.hyracks.storage.am.vector.impls.VTreeSearchCursor; import org.apache.hyracks.storage.am.vector.impls.VTreeSearchPredicate; import org.apache.hyracks.storage.am.vector.utils.VTreeDataTupleAccessor; +import org.apache.hyracks.storage.am.vector.utils.VTreeSearchParameters; import org.apache.hyracks.storage.common.EnforcedIndexCursor; import org.apache.hyracks.storage.common.ICursorInitialState; import org.apache.hyracks.storage.common.IIndexAccessParameters; @@ -205,19 +203,15 @@ // Get index access parameters IIndexAccessParameters iap = ((LSMVTreeOpContext) opCtx).getIndexAccessParameters(); - // Initialize vector accessor from factory in parameters - IVTreeBinaryAccessorFactory vectorAccessorFactory = - (IVTreeBinaryAccessorFactory) iap.getParameters().get(IVTreeBinaryAccessorFactory.IAP_KEY); - if (vectorAccessorFactory != null) { - this.vectorAccessor = vectorAccessorFactory.createAccessor(); - } + // From the index, which was configured with it at creation: it cannot be missing, so there is + // nothing here to validate or to fail on. + this.vectorAccessor = ((LSMVTree) opCtx.getIndex()).getVectorAccessorFactory().createAccessor(); // Create cluster selection strategy (minProbeFraction → nprobe + DFS fallback) this.clusterStrategy = new NprobeClusterSelectionStrategy(vectorPred.getMinProbeFraction(), epsilon); // Create spillable top-K buffer (follows inverted index pattern: pass ctx via IAP) - IHyracksTaskContext ctx = (IHyracksTaskContext) iap.getParameters().get(HyracksConstants.HYRACKS_TASK_CONTEXT); - this.topKBuffer = new SpillableTopKBuffer(candidateLimit, ctx); + this.topKBuffer = new SpillableTopKBuffer(candidateLimit, VTreeSearchParameters.requireTaskContext(iap)); // Initialize cluster tracking arrays clusterExhausted = new boolean[numComponents]; @@ -761,8 +755,13 @@ // The enforced contract guarantees doClose() already ran (destroy requires the CLOSED state), which // ended the current search. doDestroy() reclaims the per-component accessors + cursors for good, // matching LSMIndexSearchCursor.doDestroy. - Throwable failure = CleanupUtils.destroy(null, vTreeAccessors); - failure = CleanupUtils.destroy(failure, rangeCursors); + // + // Both arrays may still be null: doOpen() validates its access parameters before allocating them, + // so a cursor whose open failed reaches destroy() with nothing to reclaim. CleanupUtils.destroy + // dereferences the array, so passing null threw a NullPointerException from the cleanup path and + // replaced whatever diagnosis doOpen() had produced with a stack trace pointing at destroy(). + Throwable failure = vTreeAccessors == null ? null : CleanupUtils.destroy(null, vTreeAccessors); + failure = rangeCursors == null ? failure : CleanupUtils.destroy(failure, rangeCursors); vTreeAccessors = null; rangeCursors = null; if (failure != null) { diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTree.java b/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTree.java index 39d8112..fcc2d54 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTree.java +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTree.java @@ -50,6 +50,7 @@ import org.apache.hyracks.storage.am.vector.utils.RngAcceptanceFilter; import org.apache.hyracks.storage.am.vector.utils.VTreeMetadataKeys; import org.apache.hyracks.storage.am.vector.utils.VTreeNavigationUtils; +import org.apache.hyracks.storage.am.vector.utils.VTreeSearchParameters; import org.apache.hyracks.storage.common.IIndexAccessParameters; import org.apache.hyracks.storage.common.IIndexBulkLoader; import org.apache.hyracks.storage.common.IIndexCursor; @@ -638,12 +639,14 @@ tree.metadataFrameFactory, tree.dataFrameFactory, tree.freePageManager, tree.cmpFactories, tree.vectorDimensions, iap.getModificationCallback(), iap.getSearchOperationCallback(), tree.dataTupleBuilderFactory, tree.quantizationParams); - this.queryDistanceFunctionFactory = - (IVTreeDistanceFunctionFactory) iap.getParameters().get(IVTreeDistanceFunctionFactory.IAP_KEY); - this.binaryAccessorFactory = - (IVTreeBinaryAccessorFactory) iap.getParameters().get(IVTreeBinaryAccessorFactory.IAP_KEY); - this.quantizerFactory = (IVTreeQuantizerFactory) iap.getParameters().get(IVTreeQuantizerFactory.IAP_KEY); - this.injectedQuantizer = (IVTreeQuantizer) iap.getParameters().get(IVTreeQuantizer.IAP_KEY); + // Both come from the tree, which was configured from the same index metadata the search + // operator reads. They used to arrive through the access-parameters map as well, which meant + // the operator rebuilt values the index already held and a search could fail for want of a + // parameter that was never needed. + this.queryDistanceFunctionFactory = tree.distanceFunctionFactory; + this.binaryAccessorFactory = tree.vectorAccessorFactory; + this.quantizerFactory = VTreeSearchParameters.quantizerFactory(iap); + this.injectedQuantizer = VTreeSearchParameters.injectedQuantizer(iap); this.quantizationParams = tree.getQuantizationParams(); } @@ -733,10 +736,18 @@ vectorCursor.open(initialState, searchPred); } - /** Decode the query vector from the predicate's tuple via the IAP accessor factory (null if none). */ + /** + * Decode the query vector from the predicate's tuple. Returns {@code null} only when there is no + * query tuple to decode — an operation with no vector predicate, such as a full scan. + * <p> + * A predicate that does carry a query tuple requires the decoder, so its absence is reported here + * by key rather than passed on. {@code extractVectorFromTuple} answers {@code null} for a null + * factory, so the missing parameter used to become a null query vector and surface as a + * {@code NullPointerException} further along, in a frame with no connection to the access + * parameters that were never populated. + */ private double[] extractQueryVector(ISearchPredicate searchPred) throws HyracksDataException { if (searchPred instanceof VTreeSearchPredicate vectorPred && vectorPred.getQueryTuple() != null) { - // binaryAccessorFactory was resolved from the IAP at accessor construction. return VTreeTupleUtils.extractVectorFromTuple(vectorPred.getQueryTuple(), vectorPred.getQueryFieldIndex(), binaryAccessorFactory); } @@ -750,7 +761,7 @@ */ private IVTreeDistanceFunctionFactory resolveDistanceFunctionFactory() { // queryDistanceFunctionFactory was resolved from the IAP at accessor construction. - return queryDistanceFunctionFactory != null ? queryDistanceFunctionFactory : tree.distanceFunctionFactory; + return queryDistanceFunctionFactory; } /** Build the cursor initial state: root page (static for memory components), query vector, metric fn. */ diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/utils/VTreeSearchParameters.java b/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/utils/VTreeSearchParameters.java new file mode 100644 index 0000000..35272dd --- /dev/null +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/utils/VTreeSearchParameters.java @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.hyracks.storage.am.vector.utils; + +import org.apache.hyracks.api.context.IHyracksTaskContext; +import org.apache.hyracks.api.exceptions.ErrorCode; +import org.apache.hyracks.api.exceptions.HyracksDataException; +import org.apache.hyracks.api.util.HyracksConstants; +import org.apache.hyracks.storage.am.vector.api.IVTreeQuantizer; +import org.apache.hyracks.storage.am.vector.api.IVTreeQuantizerFactory; +import org.apache.hyracks.storage.common.IIndexAccessParameters; +import org.apache.hyracks.util.annotations.AiProvenance; + +/** + * The channel by which the query layer hands per-search state to the vector index: reads of the + * {@link IIndexAccessParameters} map, in one place and with the required/optional distinction made + * explicit. + * <p> + * The map is the platform's mechanism and is not going away — it is how the inverted index passes its + * task context too. What this fixes is the reading of it. The entries below were retrieved by unchecked + * casts out of {@code iap.getParameters()}, so a wrong type surfaced as a {@code ClassCastException} at a + * point unrelated to whoever populated the map, and an absent entry surfaced as {@code null} that was + * either ignored or dereferenced several frames later. + * <p> + * {@link IIndexAccessParameters#getParameter} already casts safely, but it returns {@code null} for + * <em>both</em> "absent" and "present with the wrong type". The <b>required</b> accessors here separate + * the two and name the key, the expected type and what it is for, so a misconfigured operator is + * diagnosable from the message alone. + * <p> + * The <b>optional</b> accessors delegate to {@code getParameter} and so keep that conflation: a + * wrong-typed optional entry reads as absent. That is deliberate rather than ideal — they are called from + * {@code VTree.VTreeAccessor}'s constructor, which serves the write path as well and whose signature is + * fixed by {@code createOpContext}, so they cannot throw a checked exception. What they do buy over the + * unchecked casts they replace is that a wrong type can no longer raise a {@code ClassCastException} in a + * frame unrelated to whoever populated the map. + * <p> + * <b>What is deliberately not here.</b> The vector accessor factory and the distance-function factory + * used to travel through this map. Both are fixed by the index — the metric is chosen at + * {@code CREATE INDEX} time and the optimizer falls back to a full scan rather than using the index for a + * query asking for a different one — and {@code LSMVTree} already holds instances built from the same + * metadata the search operator reads. They are taken from the index now, which removes two entries that + * could only ever equal what the storage layer had, and with them the possibility of a search failing for + * want of a parameter it never needed. + * <p> + * <b>The channel.</b> + * <table> + * <tr><th>key</th><th>required</th><th>set by</th></tr> + * <tr><td>{@code HYRACKS_TASK_CONTEXT}</td><td>yes, for the top-K cursor's spill buffer</td> + * <td>{@code VTreeSearchOperatorNodePushable}</td></tr> + * <tr><td>quantizer factory</td><td>no — no quantized distances when absent</td> + * <td>{@code VTreeSearchOperatorNodePushable}, for quantized indexes</td></tr> + * <tr><td>quantizer instance</td><td>no — a test seam, tried after the factory</td><td>tests</td></tr> + * </table> + * <p> + * The quantizer factory is index-fixed too and should follow, but it cannot be taken from the index the + * same way: {@code OptimizedScalarQuantizerFactory} lives in {@code asterix-common}, so this layer cannot + * build one, and routing it through the resource means persisting it on the JSON-serialized + * {@code LSMVTreeLocalResource} the way the distance-function factory already is. That is a wider change + * than taking a value the index is already holding. + * <p> + * Cursor selection ({@code USE_TOPK_SEARCH}) is deliberately not here: it is read by + * {@code LSMVTreeIndexAccessor} in the LSM layer, which is the only place that can act on it, and that + * layer is not visible from this module. + */ +@AiProvenance(agent = AiProvenance.Agent.CLAUDE_OPUS_5, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.REFACTORED, notes = "Typed reads replacing unchecked casts on the IAP map") +public final class VTreeSearchParameters { + + private VTreeSearchParameters() { + } + + /** The task context the top-K cursor's spill buffer allocates frames from. Required by that cursor. */ + public static IHyracksTaskContext requireTaskContext(IIndexAccessParameters iap) throws HyracksDataException { + return require(iap, HyracksConstants.HYRACKS_TASK_CONTEXT, IHyracksTaskContext.class, + "the top-K spill buffer's frame allocation"); + } + + /** {@code null} means quantized distances are unavailable for this search, which is a valid state. */ + public static IVTreeQuantizerFactory quantizerFactory(IIndexAccessParameters iap) { + return iap.getParameter(IVTreeQuantizerFactory.IAP_KEY, IVTreeQuantizerFactory.class); + } + + /** A pre-built quantizer, tried only after {@link #quantizerFactory}. A test seam. */ + public static IVTreeQuantizer injectedQuantizer(IIndexAccessParameters iap) { + return iap.getParameter(IVTreeQuantizer.IAP_KEY, IVTreeQuantizer.class); + } + + private static <T> T require(IIndexAccessParameters iap, String key, Class<T> type, String purpose) + throws HyracksDataException { + Object value = raw(iap, key); + if (value == null) { + throw HyracksDataException.create(ErrorCode.ILLEGAL_STATE, + "index access parameter `" + key + "` is required for " + purpose + + " but was not set; the search operator must put a " + type.getSimpleName() + + " under it before opening the cursor"); + } + if (!type.isInstance(value)) { + // Reported separately from absence: the two have different causes and different fixes, and + // getParameter() cannot tell them apart because it answers null to both. + throw HyracksDataException.create(ErrorCode.ILLEGAL_STATE, "index access parameter `" + key + "` holds a " + + value.getClass().getName() + " but " + type.getSimpleName() + " is required for " + purpose); + } + return type.cast(value); + } + + private static Object raw(IIndexAccessParameters iap, String key) { + return iap.getParameters() == null ? null : iap.getParameters().get(key); + } +} diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-vtree-test/src/test/java/org/apache/hyracks/storage/am/lsm/vector/LSMVTreeCursorAgreementTest.java b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-vtree-test/src/test/java/org/apache/hyracks/storage/am/lsm/vector/LSMVTreeCursorAgreementTest.java index 571674d..39c948c 100644 --- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-vtree-test/src/test/java/org/apache/hyracks/storage/am/lsm/vector/LSMVTreeCursorAgreementTest.java +++ b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-vtree-test/src/test/java/org/apache/hyracks/storage/am/lsm/vector/LSMVTreeCursorAgreementTest.java @@ -232,6 +232,60 @@ } } + /** + * A search opened without the parameters the top-K cursor needs must say which one is missing. + * <p> + * The task context is the one entry that genuinely has to arrive this way — it is per-operation, and + * the inverted index passes its own the same way. Omitting it used to surface as a message from deep + * inside the spill buffer, frames away from the operator that failed to populate the map. + */ + @Test + public void aTopKSearchMissingItsTaskContextNamesWhatIsMissing() throws Exception { + AbstractVectorTreeTestContext ctx = newContext(collidingDistanceCluster()); + try { + ctx.getIndex().create(); + ctx.getIndex().activate(); + testUtils.buildStaticStructure(ctx); + testUtils.bulkLoadRecords(ctx); + + HyracksDataException noContext = Assert.assertThrows(HyracksDataException.class, + () -> searchWithout(ctx, HyracksConstants.HYRACKS_TASK_CONTEXT)); + Assert.assertTrue("message should name the missing key, was: " + noContext.getMessage(), + noContext.getMessage().contains(HyracksConstants.HYRACKS_TASK_CONTEXT)); + } finally { + ctx.getIndex().deactivate(); + } + } + + /** Opens a top-K search with every required parameter except {@code omittedKey}. */ + private void searchWithout(AbstractVectorTreeTestContext ctx, String omittedKey) throws Exception { + ArrayTupleBuilder queryTupleBuilder = new ArrayTupleBuilder(1); + queryTupleBuilder.addField(DoubleArraySerializerDeserializer.INSTANCE, new double[] { 5, 0, 0 }); + ArrayTupleReference queryTuple = new ArrayTupleReference(); + queryTuple.reset(queryTupleBuilder.getFieldEndOffsets(), queryTupleBuilder.getByteArray()); + + VTreeSearchPredicate predicate = new VTreeSearchPredicate(); + predicate.setQueryTuple(queryTuple); + predicate.setQueryFieldIndex(0); + predicate.setK(K); + + IndexAccessParameters iap = + new IndexAccessParameters(TestOperationCallback.INSTANCE, TestOperationCallback.INSTANCE); + iap.getParameters().put(LSMVTreeTopKSearchCursor.IAP_KEY, Boolean.TRUE); + iap.getParameters().put(IVTreeQuantizer.IAP_KEY, NoOpVectorQuantizer.INSTANCE); + if (!HyracksConstants.HYRACKS_TASK_CONTEXT.equals(omittedKey)) { + iap.getParameters().put(HyracksConstants.HYRACKS_TASK_CONTEXT, ctx.getHyracksTaskContext()); + } + + IIndexAccessor accessor = ctx.getIndex().createAccessor(iap); + IIndexCursor cursor = accessor.createSearchCursor(false); + try { + accessor.search(cursor, predicate); + } finally { + cursor.destroy(); + } + } + // ---- the agreement assertion -------------------------------------------------------------- /** -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21682?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I41fdd9e8c380b93d45c2626f14079c32a9f0d060 Gerrit-Change-Number: 21682 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman <[email protected]>
