>From Ali Alsuliman <[email protected]>: Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21678?usp=email )
Change subject: [ASTERIXDB-3817][STO] Hold the borrowed static structure in one ref ...................................................................... [ASTERIXDB-3817][STO] Hold the borrowed static structure in one ref A memory component has no clustering structure of its own and navigates the immutable one owned by the static disk component. That arrangement was expressed as six plain fields plus a boolean, and "am I attached?" was re-derived from them at sixteen call sites. Two of those fields were tested independently: twelve sites keyed on staticBufferCache and four on centroidDirPageMap. resetInitialization() nulled the map but left the borrowed buffer cache, file id and root page in place, so a recycled component read as detached by the four and as still attached by the twelve -- navigating a static structure it had already let go of. Gathering the state into StaticStructureRef and holding a single reference makes attachment atomic, so the two views cannot disagree and resetInitialization() is one write. Publication no longer rests on prose. The record's fields are final and the reference is published by one volatile write, so a reader sees either no attachment or a complete one. That replaces an eleven-line comment arguing the fields were safe unvolatile because the LSM operation tracker supplies the happens-before -- an argument every future reader had to re-verify. It costs one volatile read per navigation, and each navigation now reads it once into a local instead of re-reading a field up to four times, so a single operation can no longer mix an attached and a detached view of the tree. directoryPageFor() replaces four copies of the same centroid-id arithmetic, in findClosestClusterFromRoot, findCloseCentroidsLevelWiseGlobalSortFromRoot, VTreeSearchCursor.getMetadataPageIdFromCluster and VTreeFlushLoader. With those gone the directory mapping is no longer handed out as a mutable int[]: the cursor holds the ref, VTreeFlushLoader takes it through a package-private accessor, and the public getCentroidDirPageMap() is deleted. getNumLeafCentroidMem/getFirstLeafCentroidIdMem stay public for LSMVTree, which is in another package. The number of branch tests is unchanged -- a memory component borrows and a disk component owns, so something must test which. What is removed is the divergent predicates, the duplicated arithmetic and the documented threading contract. 31 LSM VTree tests and 19 asterixdb vector runtime tests pass, the multithread test among them. None of them exercises the recycle-then- navigate window the old inconsistency lived in; covering that wants a standalone vtree test module, which does not exist yet. Ext-ref: MB-73194 Co-Authored-By: Claude Opus 5 <[email protected]> Change-Id: I3fee713442c1143e67fc9931208a0df9dca9d597 --- A hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/StaticStructureRef.java M hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTree.java M hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTreeFlushLoader.java M hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTreeSearchCursor.java 4 files changed, 197 insertions(+), 115 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/78/21678/1 diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/StaticStructureRef.java b/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/StaticStructureRef.java new file mode 100644 index 0000000..8fc3aaf --- /dev/null +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/StaticStructureRef.java @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.hyracks.storage.am.vector.impls; + +import java.util.Objects; + +import org.apache.hyracks.storage.common.buffercache.IBufferCache; +import org.apache.hyracks.util.annotations.AiProvenance; + +/** + * Everything a memory component needs in order to navigate a <em>different</em> component's static + * clustering structure: that component's buffer cache, file id and root page, plus the + * centroid→directory-page mapping allocated in this component's own virtual buffer cache. + * <p> + * A memory component has no clustering structure of its own — it borrows the immutable one from the + * static disk component — which is the central design decision of this module. Holding it as one + * immutable object rather than six loose fields makes that arrangement enforceable in two ways. + * <p> + * <b>Attachment becomes atomic.</b> "Am I attached?" is one reference test, so the state cannot be + * half-cleared. It previously could: {@code resetInitialization()} nulled the directory map but left the + * borrowed buffer cache, file id and root page in place, so a recycled component read as detached by the + * checks keyed on the map and as still-attached by the twelve keyed on the buffer cache — pointing at a + * static structure it had already let go of. + * <p> + * <b>Publication stops depending on prose.</b> Every field here is final, so a reader that sees the + * reference sees fully-initialized contents. The reference itself is published by a single volatile + * write, which is what makes this safe on its own rather than by an argument about the LSM operation + * tracker's happens-before. The cost is one volatile read per navigation in place of several plain + * reads of fields that were only safe by that argument. + * + * @param bufferCache the static structure's buffer cache, read-only from here + * @param fileId the static structure's file id + * @param rootPageId the static structure's root page + * @param centroidDirPageMap leaf-centroid index → directory page id, in this component's own cache + * @param firstLeafCentroidId centroid id that {@code centroidDirPageMap[0]} corresponds to + * @param numLeafCentroids number of leaf centroids in the static structure + */ +@AiProvenance(agent = AiProvenance.Agent.CLAUDE_OPUS_5, tool = AiProvenance.Tool.CLAUDE_CODE_UI, + contributionKind = AiProvenance.ContributionKind.REFACTORED, + notes = "Replaces six loose VTree fields") +record StaticStructureRef(IBufferCache bufferCache, int fileId, int rootPageId, int[] centroidDirPageMap, + int firstLeafCentroidId, int numLeafCentroids) { + + /** + * Returned by {@link #directoryPageFor(int)} for a centroid the mapping does not cover. Safe as a + * sentinel because every entry comes from {@code IPageManager#takePage}, which never returns a + * negative page id. + */ + static final long NO_DIRECTORY_PAGE = -1; + + StaticStructureRef { + Objects.requireNonNull(bufferCache, "bufferCache"); + Objects.requireNonNull(centroidDirPageMap, "centroidDirPageMap"); + } + + /** + * The directory page holding a leaf centroid's data pages, or {@link #NO_DIRECTORY_PAGE} when the + * centroid id falls outside this mapping — which is not an error: a caller that navigated the static + * structure can legitimately land on a centroid this component never allocated a directory for, and + * every caller falls back to its own resolution in that case. + */ + long directoryPageFor(int centroidId) { + int index = centroidId - firstLeafCentroidId; + return index >= 0 && index < centroidDirPageMap.length ? centroidDirPageMap[index] : NO_DIRECTORY_PAGE; + } +} diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTree.java b/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTree.java index f829b83..3154ece 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTree.java +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTree.java @@ -100,27 +100,19 @@ */ private final VTreePageMutator pageMutator; - // Static-structure navigation state (memory components only). Threading contract: this group is - // written exactly once by setStaticStructure() (idempotent via the `initialized` guard) while a - // memory component is being allocated or recycled — i.e. before that component is published for - // operations — and is read-only thereafter until resetInitialization() clears it for the next - // recycle. Publication to the threads that later run searches/inserts on the component is provided - // by the LSM harness's operation-tracker happens-before (a thread must enter the component through - // the tracker before touching it), the same mechanism that publishes all other in-memory component - // state (e.g. BTree memory frames). These fields are therefore intentionally NOT volatile: adding - // volatility would tax the per-navigation read of staticBufferCache without adding a guarantee the - // harness does not already give. Do not read this group outside that established happens-before. - private boolean initialized = false; - - // For memory components: reference to static structure for navigation - private IBufferCache staticBufferCache; - private int staticFileId; - private int staticRootPage; - - // Centroid-to-directory-page mapping (memory components only) - private int[] centroidDirPageMap; // centroidIndex -> VBC directory page ID - private int firstLeafCentroidIdMem; - private int numLeafCentroidMem; + /** + * The static structure this memory component navigates through, or {@code null} for a disk component + * (which has its own) and for a memory component between recycle and re-attach. Written by + * {@link #setStaticStructure} while the component is being allocated or recycled — before it is + * published for operations — and cleared by {@link #resetInitialization}; read on every navigation. + * <p> + * {@code volatile} because it is the whole of the attachment state: a reader either sees a fully + * built {@link StaticStructureRef} or sees {@code null}, with no window in which some of the borrowed + * state is visible and some is not. That replaces the previous arrangement of six plain fields whose + * safety rested on the LSM operation tracker's happens-before, documented in an eleven-line comment + * that every future reader had to re-verify. One volatile read per navigation is the price. + */ + private volatile StaticStructureRef staticStructure; public VTree(IBufferCache bufferCache, IPageManager freePageManager, ITreeIndexFrameFactory interiorFrameFactory, ITreeIndexFrameFactory leafFrameFactory, ITreeIndexFrameFactory metadataFrameFactory, @@ -302,25 +294,26 @@ public ClusterSearchResult findClosestClusterFromRoot(double[] queryVector, IVTreeDistanceFunction distanceFunction, double[] quantizedQueryVector, IVTreeQuantizer quantizer) throws HyracksDataException { - // For memory components: navigate via static structure reference - IBufferCache navBC = (staticBufferCache != null) ? staticBufferCache : bufferCache; - int navFileId = (staticBufferCache != null) ? staticFileId : getFileId(); - int navRoot = (staticBufferCache != null) ? staticRootPage : rootPage; + // Read the attachment once: a memory component navigates the static structure it borrowed, a + // disk component its own tree. Re-reading the volatile per use could otherwise mix the two. + StaticStructureRef ref = staticStructure; + IBufferCache navBC = ref != null ? ref.bufferCache() : bufferCache; + int navFileId = ref != null ? ref.fileId() : getFileId(); + int navRoot = ref != null ? ref.rootPageId() : rootPage; LOGGER.log(Level.TRACE, "Starting findClosestClusterFromRoot with navRoot={}, isMemoryComponent={}", navRoot, - staticBufferCache != null); + ref != null); ClusterSearchResult result = VTreeNavigationUtils.findClosestCentroid(navBC, navFileId, navRoot, getInteriorFrameFactory(), getLeafFrameFactory(), queryVector, distanceFunction, quantizedQueryVector, quantizer); // For memory components: replace directoryPageId with VBC mapping - if (centroidDirPageMap != null) { - int centroidIndex = result.centroidId - firstLeafCentroidIdMem; - if (centroidIndex >= 0 && centroidIndex < centroidDirPageMap.length) { + if (ref != null) { + long dirPageId = ref.directoryPageFor(result.centroidId); + if (dirPageId != StaticStructureRef.NO_DIRECTORY_PAGE) { result = ClusterSearchResult.create(result.leafPageId, result.clusterIndex, result.centroid, - result.distance, result.centroidId, centroidDirPageMap[centroidIndex], - result.quantizedDistance); + result.distance, result.centroidId, dirPageId, result.quantizedDistance); } } @@ -337,7 +330,8 @@ * For disk components, this returns the component's own buffer cache. */ public IBufferCache getNavigationBufferCache() { - return (staticBufferCache != null) ? staticBufferCache : bufferCache; + StaticStructureRef ref = staticStructure; + return ref != null ? ref.bufferCache() : bufferCache; } /** @@ -346,7 +340,8 @@ * For disk components, this returns the component's own file ID. */ public int getNavigationFileId() { - return (staticBufferCache != null) ? staticFileId : getFileId(); + StaticStructureRef ref = staticStructure; + return ref != null ? ref.fileId() : getFileId(); } /** @@ -355,7 +350,8 @@ * For disk components, this returns the component's own root page. */ public int getNavigationRootPageId() { - return (staticBufferCache != null) ? staticRootPage : rootPage; + StaticStructureRef ref = staticStructure; + return ref != null ? ref.rootPageId() : rootPage; } /** @@ -366,10 +362,10 @@ public List<ClusterSearchResult> findCloseCentroidsLevelWiseGlobalSortFromRoot(double[] queryVector, IVTreeDistanceFunction distanceFunction, double ep) throws HyracksDataException { - // For memory components: navigate via static structure reference - IBufferCache navBC = (staticBufferCache != null) ? staticBufferCache : bufferCache; - int navFileId = (staticBufferCache != null) ? staticFileId : getFileId(); - int navRoot = (staticBufferCache != null) ? staticRootPage : rootPage; + StaticStructureRef ref = staticStructure; + IBufferCache navBC = ref != null ? ref.bufferCache() : bufferCache; + int navFileId = ref != null ? ref.fileId() : getFileId(); + int navRoot = ref != null ? ref.rootPageId() : rootPage; LOGGER.log(Level.TRACE, "Starting findCloseCentroidsLevelWiseFromRoot with navRoot={}", navRoot); @@ -377,15 +373,13 @@ navRoot, getInteriorFrameFactory(), getLeafFrameFactory(), queryVector, distanceFunction, ep); // For memory components: replace directoryPageId with VBC mapping - if (centroidDirPageMap != null) { + if (ref != null) { for (int r = 0; r < results.size(); r++) { ClusterSearchResult result = results.get(r); - int centroidIndex = result.centroidId - firstLeafCentroidIdMem; - if (centroidIndex >= 0 && centroidIndex < centroidDirPageMap.length) { - results.set(r, - ClusterSearchResult.create(result.leafPageId, result.clusterIndex, result.centroid, - result.distance, result.centroidId, centroidDirPageMap[centroidIndex], - result.quantizedDistance)); + long dirPageId = ref.directoryPageFor(result.centroidId); + if (dirPageId != StaticStructureRef.NO_DIRECTORY_PAGE) { + results.set(r, ClusterSearchResult.create(result.leafPageId, result.clusterIndex, result.centroid, + result.distance, result.centroidId, dirPageId, result.quantizedDistance)); } } } @@ -402,59 +396,61 @@ } public boolean isInitialized() { - return initialized; + return staticStructure != null; } /** - * Reset initialization state so that static structure directory pages - * can be re-created after a memory component flush/recycle. + * Detach from the static structure so that its directory pages are re-created when this memory + * component is next allocated. One write clears the whole attachment — previously the directory map + * was nulled while the borrowed buffer cache, file id and root page were left behind, so a recycled + * component still read as attached to every check keyed on the buffer cache. */ public void resetInitialization() { - initialized = false; - centroidDirPageMap = null; + staticStructure = null; } /** - * Initialize this memory component's static-structure navigation state (see the field-group comment - * above for the full threading contract). Called by the LSM layer during memory-component allocation - * or post-flush recycle, before the component is published for operations; {@code synchronized} plus - * the {@code initialized} guard make it a safe, idempotent single write. Reads of the fields it sets - * rely on the LSM harness's operation-tracker happens-before for visibility, so callers must not read - * them concurrently with this method outside that ordering. + * Attach this memory component to the static structure it will navigate. Called by the LSM layer + * during memory-component allocation or post-flush recycle, before the component is published for + * operations; {@code synchronized} plus the null check make it idempotent, and every field the + * attachment consists of is gathered into a {@link StaticStructureRef} that is published by the + * single volatile write on the last line. A reader therefore sees either no attachment or a complete + * one, without relying on the operation tracker's ordering to hide a partial write. */ public synchronized void setStaticStructure(VTreeAccessor staticAccessor) throws HyracksDataException { - if (initialized) { - return; // Already initialized, skip + if (staticStructure != null) { + return; // Already attached, skip } - VTree staticStructure = staticAccessor.getIndex(); + VTree source = staticAccessor.getIndex(); ITreeIndexMetadataFrame metaFrame = staticAccessor.getOpContext().getMetaFrame(); - // Store references to static structure for read-only navigation - this.staticBufferCache = staticStructure.getBufferCache(); - this.staticFileId = staticStructure.getFileId(); - this.staticRootPage = staticStructure.rootPage; + // Captured into locals and published as one object at the end of this method, so no thread can + // observe a partly-attached component. + IBufferCache staticCache = source.getBufferCache(); + int staticFileId = source.getFileId(); + int staticRootPage = source.rootPage; // Pin the static structure's metadata page onto metaFrame before reading // (getMaxPageId internally calls metaFrame.setPage() which initializes the frame's buffer) - staticStructure.getPageManager().getMaxPageId(metaFrame); + source.getPageManager().getMaxPageId(metaFrame); // Read metadata from static structure LongPointable value1 = LongPointable.FACTORY.createPointable(); LongPointable value2 = LongPointable.FACTORY.createPointable(); metaFrame.get(VTreeMetadataKeys.NUM_LEAF_CENTROIDS, value1); metaFrame.get(VTreeMetadataKeys.FIRST_LEAF_CENTROID_ID, value2); - this.numLeafCentroidMem = value1.intValue(); - this.firstLeafCentroidIdMem = value2.intValue(); + int numLeafCentroids = value1.intValue(); + int firstLeafCentroidId = value2.intValue(); // Create empty directory pages in VBC (using takePage() directly) ITreeIndexMetadataFrame vbcMetaFrame = freePageManager.createMetadataFrame(); ITreeIndexFrame directoryFrame = metadataFrameFactory.createFrame(); - centroidDirPageMap = new int[numLeafCentroidMem]; + int[] dirPageMap = new int[numLeafCentroids]; - for (int i = 0; i < numLeafCentroidMem; i++) { + for (int i = 0; i < numLeafCentroids; i++) { int dirPageId = freePageManager.takePage(vbcMetaFrame); - centroidDirPageMap[i] = dirPageId; + dirPageMap[i] = dirPageId; ICachedPage targetPage = bufferCache.pin(BufferedFileHandle.getDiskPageId(getFileId(), dirPageId), NEW); try { @@ -469,7 +465,8 @@ LOGGER.log(Level.TRACE, "Created directory page {} for leaf centroid {}", dirPageId, i); } - initialized = true; + this.staticStructure = new StaticStructureRef(staticCache, staticFileId, staticRootPage, dirPageMap, + firstLeafCentroidId, numLeafCentroids); } /** @@ -494,28 +491,33 @@ * away or, worse, a silent "no centroids to copy". Both hide the actual fault, which is a caller * using the component before it was attached or after it was recycled. */ - private void requireAttachedStaticStructure(String what) throws HyracksDataException { - if (!initialized) { + private StaticStructureRef requireAttachedStaticStructure(String what) throws HyracksDataException { + StaticStructureRef ref = staticStructure; + if (ref == null) { throw HyracksDataException.create(ErrorCode.ILLEGAL_STATE, "read of " + what + " on a VTree with no attached static structure: setStaticStructure() has not run, or" + " resetInitialization() has already recycled this component"); } + return ref; } - public int[] getCentroidDirPageMap() throws HyracksDataException { - requireAttachedStaticStructure("centroidDirPageMap"); - return centroidDirPageMap; + /** + * The attachment itself, for same-package collaborators that need more than one part of it. Returning + * the immutable ref rather than the {@code int[]} keeps the directory mapping from being handed out as + * a mutable array, and keeps the centroid-id arithmetic in one place + * ({@link StaticStructureRef#directoryPageFor}). + */ + StaticStructureRef requireStaticStructure() throws HyracksDataException { + return requireAttachedStaticStructure("staticStructure"); } public int getFirstLeafCentroidIdMem() throws HyracksDataException { - requireAttachedStaticStructure("firstLeafCentroidIdMem"); - return firstLeafCentroidIdMem; + return requireAttachedStaticStructure("firstLeafCentroidIdMem").firstLeafCentroidId(); } public int getNumLeafCentroidMem() throws HyracksDataException { - requireAttachedStaticStructure("numLeafCentroidMem"); - return numLeafCentroidMem; + return requireAttachedStaticStructure("numLeafCentroidMem").numLeafCentroids(); } /** @@ -580,14 +582,14 @@ * Prepare data-page access for a single (already resolved) cluster: pin/latch the leaf page when needed * and resolve the metadata page id. Used by insert, delete, and update once per replica cluster. * <ul> - * <li>Memory components ({@code centroidDirPageMap != null}): the metadata page is taken straight from + * <li>Memory components (an attached static structure): the metadata page is taken straight from * the VBC centroid→directory mapping carried on {@code clusterResult}; no leaf page is pinned.</li> * <li>Disk components: the leaf page is write-latched and the metadata pointer read from its frame.</li> * </ul> */ private ClusterAccessResult prepareClusterAccess(ClusterSearchResult clusterResult, VTreeOpContext ctx) throws HyracksDataException { - if (centroidDirPageMap != null) { + if (staticStructure != null) { return new ClusterAccessResult(clusterResult, null, clusterResult.directoryPageId, bufferCache); } @@ -686,12 +688,13 @@ } private void configureCursor(VTreeSearchCursor cursor) { - if (tree.staticBufferCache != null) { - cursor.setBufferCache(tree.staticBufferCache); - cursor.setFileId(tree.staticFileId); - cursor.setRootPageId(tree.staticRootPage); + StaticStructureRef ref = tree.staticStructure; + if (ref != null) { + cursor.setBufferCache(ref.bufferCache()); + cursor.setFileId(ref.fileId()); + cursor.setRootPageId(ref.rootPageId()); cursor.setDataBufferCache(tree.bufferCache, tree.getFileId()); - cursor.setCentroidDirPageMap(tree.centroidDirPageMap, tree.firstLeafCentroidIdMem); + cursor.setStaticStructure(ref); } else { cursor.setBufferCache(tree.bufferCache); cursor.setFileId(tree.getFileId()); @@ -749,9 +752,9 @@ private VTreeCursorInitialState buildInitialState(double[] queryVector, IVTreeDistanceFunction distanceFunction) { VTreeCursorInitialState initialState = new VTreeCursorInitialState(ctx.getAccessor()); - // For memory components, use staticRootPage (the static structure's root); - // for disk components, use the tree's own rootPage - initialState.setRootPageId(tree.staticBufferCache != null ? tree.staticRootPage : tree.rootPage); + // For memory components, the borrowed static structure's root; for disk components, our own + StaticStructureRef ref = tree.staticStructure; + initialState.setRootPageId(ref != null ? ref.rootPageId() : tree.rootPage); if (queryVector != null) { initialState.setQueryVector(queryVector); } diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTreeFlushLoader.java b/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTreeFlushLoader.java index 7670dbe..cdba5e2 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTreeFlushLoader.java +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTreeFlushLoader.java @@ -93,7 +93,7 @@ /** * Copy static structure pages to end of file with pointer adjustment. * Interior child pointers are offset by staticBasePageId. - * Leaf metadata pointers are set from the source memory tree's centroidDirPageMap + * Leaf metadata pointers are set from the source memory tree's centroid→directory mapping * (identity mapping: VBC page IDs = disk page IDs). * Leaf next-page pointers are offset by staticBasePageId. * <p> @@ -134,9 +134,7 @@ IVTreeInteriorFrame intFrame = (IVTreeInteriorFrame) treeIndex.getInteriorFrameFactory().createFrame(); IVTreeLeafFrame lfFrame = (IVTreeLeafFrame) treeIndex.getLeafFrameFactory().createFrame(); - int[] centroidDirPageMap = sourceMemoryTree.getCentroidDirPageMap(); - int numLeafCentroid = sourceMemoryTree.getNumLeafCentroidMem(); - int firstLeafCid = sourceMemoryTree.getFirstLeafCentroidIdMem(); + StaticStructureRef staticStructure = sourceMemoryTree.requireStaticStructure(); // Copy one source/destination page pair at a time: pin source, copy into the confiscated // destination page, release the source, patch pointers, write, then move on. @@ -168,16 +166,15 @@ intFrame.setNextPage(intFrame.getNextPage() + staticBasePageId); } } else { - // Leaf page: set metadata pointers to VBC directory page IDs - // (identity mapping means VBC page IDs = disk page IDs). Index - // centroidDirPageMap by the slot's centroid_id (cid - firstLeafCid), not by - // traversal order, since physical page-id order need not match the nextLeaf chain. + // Leaf page: set metadata pointers to VBC directory page IDs (identity mapping means VBC + // page IDs = disk page IDs). Looked up by the slot's own centroid_id, not by traversal + // order, since physical page-id order need not match the nextLeaf chain. A centroid the + // mapping does not cover keeps whatever pointer the static structure carried. lfFrame.setPage(page); for (int t = 0; t < lfFrame.getTupleCount(); t++) { - int cid = lfFrame.getCentroidId(t); - int idx = cid - firstLeafCid; - if (idx >= 0 && idx < numLeafCentroid) { - lfFrame.setMetadataPagePointer(t, centroidDirPageMap[idx]); + long dirPageId = staticStructure.directoryPageFor(lfFrame.getCentroidId(t)); + if (dirPageId != StaticStructureRef.NO_DIRECTORY_PAGE) { + lfFrame.setMetadataPagePointer(t, (int) dirPageId); } } // Offset the next-leaf pointer. The next-leaf field is dual-purpose: with the overflow flag diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTreeSearchCursor.java b/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTreeSearchCursor.java index 8a1e1ae..7cf59a9 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTreeSearchCursor.java +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VTreeSearchCursor.java @@ -72,10 +72,10 @@ private IBufferCache dataBufferCache; private int dataFileId; // Centroid-to-directory-page mapping (memory components only, null for disk) - private int[] centroidDirPageMap; - private int firstLeafCentroidIdForMap; + /** Set for memory components only; null for a disk component, which resolves locally instead. */ + private StaticStructureRef staticStructure; // Lazily-built centroid→directoryPageId map for disk components. - // When centroidDirPageMap is null and openClusterByResult() is called, + // When staticStructure is null and openClusterByResult() is called, // the ClusterSearchResult's directoryPageId may be from a DIFFERENT component // (e.g., static structure with predicted IDs). This map resolves the correct // directory page ID by scanning this component's own leaf pages. @@ -183,9 +183,8 @@ * Set centroid-to-directory-page mapping for memory components. * When set, the cursor resolves directory pages from the map instead of reading leaf pages. */ - public void setCentroidDirPageMap(int[] map, int firstLeafCentroidId) { - this.centroidDirPageMap = map; - this.firstLeafCentroidIdForMap = firstLeafCentroidId; + public void setStaticStructure(StaticStructureRef staticStructure) { + this.staticStructure = staticStructure; } public void setRootPageId(int rootPageId) { @@ -671,7 +670,7 @@ // Always resolve directoryPageId locally for this component. // The cluster's directoryPageId may come from a different LSM component // (e.g., memory component VBC page IDs vs disk component page IDs). - // getMetadataPageIdFromCluster handles both memory (centroidDirPageMap) + // getMetadataPageIdFromCluster handles both memory (the static structure's mapping) // and disk (leaf page traversal) correctly. long localDirPageId = getMetadataPageIdFromCluster(cluster); openClusterByDirectoryPage(localDirPageId); @@ -886,8 +885,8 @@ /** * Get metadata page ID from cluster search result. * - * For memory components: uses centroidDirPageMap for O(1) lookup (the map - * translates centroid IDs to VBC directory page IDs). + * For memory components: uses the borrowed static structure's centroid→directory-page + * mapping for an O(1) lookup. * * For disk components: builds a lazy local map by scanning this component's * own leaf pages. This is necessary because the ClusterSearchResult's @@ -896,11 +895,11 @@ * directory page IDs that don't match this disk component's actual IDs). */ private long getMetadataPageIdFromCluster(ClusterSearchResult clusterResult) throws HyracksDataException { - // Memory components: use centroidDirPageMap for O(1) lookup - if (centroidDirPageMap != null) { - int centroidIndex = clusterResult.centroidId - firstLeafCentroidIdForMap; - if (centroidIndex >= 0 && centroidIndex < centroidDirPageMap.length) { - return centroidDirPageMap[centroidIndex]; + // Memory components: the borrowed static structure's mapping gives an O(1) lookup + if (staticStructure != null) { + long dirPageId = staticStructure.directoryPageFor(clusterResult.centroidId); + if (dirPageId != StaticStructureRef.NO_DIRECTORY_PAGE) { + return dirPageId; } } -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21678?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I3fee713442c1143e67fc9931208a0df9dca9d597 Gerrit-Change-Number: 21678 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman <[email protected]>
