>From Ritik Raj <[email protected]>: Ritik Raj has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21624?usp=email )
Change subject: [NO ISSUE][RT] Count index pages from the file, not byte size ...................................................................... [NO ISSUE][RT] Count index pages from the file, not byte size - user model changes: no - storage format changes: no - interface changes: no Details: - DatasetStreamStatsOperatorDescriptor derived an index's page count as componentSize / pageSize. That identity holds only for an uncompressed index: on a compressed one componentSize is the compressed byte count, so the quotient under-counts, and a component whose compressed bytes are smaller than a single page floors to zero. - The counts land in the sample index's IndexStats, which the CBO reads for index-access costing, so a zero makes an index look free. - This is unreachable today only because two facts line up: the primary index is the sole compressed index, and SampleOperationsHelper gathers only the secondary indexes. The code therefore carried an undocumented dependency on every index it measures being uncompressed. - Ask the buffer cache for the page count instead. BufferedFileHandle divides the file size by the page size with header, and the compressed handle returns its look-aside file's entry count, so the result is compression-invariant -- which is what this statistic should always have been, since compression changes how many bytes a page occupies and never how many pages exist. - Counting only the tree file also excludes the bloom filter by construction, replacing the subtraction of its file length. Tests: - ddl/analyze-dataset-with-indexes passes against its unchanged expected page counts, in both the default and cloud-storage configurations, confirming the corrected count matches the previous one for uncompressed indexes. Co-Authored-By: Claude Opus 5 <[email protected]> Change-Id: Ic7885cc1688999cb5e4d67049226ad9c318710ef --- M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/DatasetStreamStatsOperatorDescriptor.java 1 file changed, 30 insertions(+), 6 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/24/21624/1 diff --git a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/DatasetStreamStatsOperatorDescriptor.java b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/DatasetStreamStatsOperatorDescriptor.java index 699fd3b..6fc9a64 100644 --- a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/DatasetStreamStatsOperatorDescriptor.java +++ b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/DatasetStreamStatsOperatorDescriptor.java @@ -43,10 +43,12 @@ import org.apache.hyracks.dataflow.std.base.AbstractSingleActivityOperatorDescriptor; import org.apache.hyracks.dataflow.std.base.AbstractUnaryInputUnaryOutputOperatorNodePushable; import org.apache.hyracks.storage.am.common.api.IIndexDataflowHelper; +import org.apache.hyracks.storage.am.common.api.ITreeIndex; import org.apache.hyracks.storage.am.common.dataflow.IIndexDataflowHelperFactory; import org.apache.hyracks.storage.am.lsm.common.api.AbstractLSMWithBloomFilterDiskComponent; import org.apache.hyracks.storage.am.lsm.common.api.ILSMDiskComponent; import org.apache.hyracks.storage.am.lsm.common.api.ILSMIndex; +import org.apache.hyracks.storage.common.IIndex; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; @@ -161,12 +163,7 @@ long numPages = 0; synchronized (indexInstance.getOperationTracker()) { for (ILSMDiskComponent component : indexInstance.getDiskComponents()) { - long componentSize = component.getComponentSize(); - if (component instanceof AbstractLSMWithBloomFilterDiskComponent) { - componentSize -= ((AbstractLSMWithBloomFilterDiskComponent) component) - .getBloomFilter().getFileReference().getFile().length(); - } - numPages += componentSize / indexInstance.getBufferCache().getPageSize(); + numPages += numPagesOf(component, indexInstance); } } IndexStats indexStats = indexesStats.computeIfAbsent(indexesNames[i], @@ -178,6 +175,33 @@ } } } + + /** + * Page count of one disk component, asked of the file rather than derived from its byte + * size. Dividing bytes by the page size only holds for an uncompressed index: on a + * compressed one the bytes on disk are the compressed bytes, so the quotient under-counts, + * and for a component smaller than one page's worth of compressed bytes it floors to zero. + * That silently reported no pages to the CBO. It stayed hidden because only the primary + * index was ever compressed, and only secondary indexes are gathered here. + * <p> + * {@code getNumPagesOfFile} is correct either way -- a plain file divides its size by the + * page size with header, a compressed one reports the entry count of its look-aside file. + * It also counts only the tree file, so the bloom filter is excluded by construction rather + * than by subtracting its file length. + */ + private long numPagesOf(ILSMDiskComponent component, ILSMIndex indexInstance) throws HyracksDataException { + IIndex componentIndex = component.getIndex(); + if (componentIndex instanceof ITreeIndex) { + return indexInstance.getBufferCache().getNumPagesOfFile(((ITreeIndex) componentIndex).getFileId()); + } + // Not a tree component: fall back to the byte estimate, minus the bloom filter as before. + long componentSize = component.getComponentSize(); + if (component instanceof AbstractLSMWithBloomFilterDiskComponent) { + componentSize -= ((AbstractLSMWithBloomFilterDiskComponent) component).getBloomFilter() + .getFileReference().getFile().length(); + } + return componentSize / indexInstance.getBufferCache().getPageSize(); + } }; } } -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21624?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ic7885cc1688999cb5e4d67049226ad9c318710ef Gerrit-Change-Number: 21624 Gerrit-PatchSet: 1 Gerrit-Owner: Ritik Raj <[email protected]>
