[
https://issues.apache.org/jira/browse/ASTERIXDB-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17948305#comment-17948305
]
ASF subversion and git services commented on ASTERIXDB-3597:
------------------------------------------------------------
Commit bf1dd860e132c067081ae2c4c576c996792510d6 in asterixdb's branch
refs/heads/master from Ritik Raj
[ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=bf1dd860e1 ]
[ASTERIXDB-3597][STO] Using interior frame slot size
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
Previously, we use the leaf frame's slot size for
interior frames, which is set as 0 for column
leaf frame. This caused underestimation of space
when writing tuples into interior frames (guide nodes).
Tuples are written left to right, while slot metadata is
written right to left. The 4-byte underestimation per tuple
could lead the frame to incorrectly believe it can
accommodate a tuple. This may cause tuple data to overlap
with the slot region, resulting in corruption.
This change uses interiorSlotSize for writing interior
row guide nodes, fixing the underestimation.
Ext-ref: MB-66227
Change-Id: Id734699118713f4754fb3ef0b93fdb9b314b448b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19645
Integration-Tests: Jenkins <[email protected]>
Tested-by: Jenkins <[email protected]>
Reviewed-by: Peeyush Gupta <[email protected]>
> Merge failure for collections with column format
> ------------------------------------------------
>
> Key: ASTERIXDB-3597
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-3597
> Project: Apache AsterixDB
> Issue Type: Bug
> Components: STO - Storage
> Affects Versions: 0.9.10
> Reporter: Ritik Raj
> Assignee: Ritik Raj
> Priority: Major
> Labels: triaged
> Fix For: 0.9.10
>
>
> While compacting the disk components, the following error was observed
> causing the MERGE operation to fail
> {code:java}
> 2025-04-09T15:57:20.934+00:00 ERRO CBAS.impls.LSMHarness
> [Executor-703:ec981dab551d6a83a8b7449f6127857f] MERGE operation {"fileName":
> "45_78_b", "ioOpID": 660815175} failed on {"dir" :
> "/var/cb-cache/@analytics/v_iodevice_5/storage/partition_69/Default/Default/users/0/users",
> "memory" : [{"state":"INACTIVE", "writers":0, "readers":0,
> "pendingFlushes":0, "id":"[136,136]", "index":{"class": "BTree", "file":
> "storage/partition_69/Default/Default/users/0/users_virtual_0"}},
> {"state":"INACTIVE", "writers":0, "readers":0, "pendingFlushes":0,
> "id":"[135,135]", "index":{"class": "BTree", "file":
> "storage/partition_69/Default/Default/users/0/users_virtual_1"}}], "disk" :
> 15, "num-scheduled-flushes":0, "current-memory-component":1}
> java.lang.ArrayIndexOutOfBoundsException: Index 1948764105 out of bounds for
> length 131072
> at
> org.apache.hyracks.storage.am.common.util.BitOperationUtils.getBit(BitOperationUtils.java:32)
> ~[hyracks-storage-am-common-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.btree.tuples.LSMBTreeTupleReference.isAntimatter(LSMBTreeTupleReference.java:92)
> ~[hyracks-storage-am-lsm-btree-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.btree.tuples.LSMBTreeTupleReference.resetByTupleOffset(LSMBTreeTupleReference.java:61)
> ~[hyracks-storage-am-lsm-btree-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.btree.tuples.LSMBTreeTupleReference.resetByTupleIndex(LSMBTreeTupleReference.java:76)
> ~[hyracks-storage-am-lsm-btree-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.btree.impls.BTreeNSMBulkLoader.propagateBulk(BTreeNSMBulkLoader.java:163)
> ~[hyracks-storage-am-btree-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.btree.column.impls.btree.ColumnBTreeBulkloader.writeFullLeafPage(ColumnBTreeBulkloader.java:162)
> ~[hyracks-storage-am-lsm-btree-column-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.btree.column.impls.btree.ColumnBTreeBulkloader.add(ColumnBTreeBulkloader.java:88)
> ~[hyracks-storage-am-lsm-btree-column-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.common.impls.LSMIndexBulkLoader.add(LSMIndexBulkLoader.java:55)
> ~[hyracks-storage-am-lsm-common-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.common.impls.ChainedLSMDiskComponentBulkLoader.add(ChainedLSMDiskComponentBulkLoader.java:68)
> ~[hyracks-storage-am-lsm-common-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.doMerge(LSMBTree.java:333)
> ~[hyracks-storage-am-lsm-btree-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.common.impls.AbstractLSMIndex.merge(AbstractLSMIndex.java:930)
> ~[hyracks-storage-am-lsm-common-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.doIo(LSMHarness.java:553)
> ~[hyracks-storage-am-lsm-common-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.merge(LSMHarness.java:593)
> ~[hyracks-storage-am-lsm-common-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.merge(LSMTreeIndexAccessor.java:129)
> ~[hyracks-storage-am-lsm-common-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.common.impls.MergeOperation.call(MergeOperation.java:52)
> ~[hyracks-storage-am-lsm-common-1.1.0-1238.jar:1.1.0-1238]
> at
> org.apache.hyracks.storage.am.lsm.common.impls.MergeOperation.call(MergeOperation.java:33)
> ~[hyracks-storage-am-lsm-common-1.1.0-1238.jar:1.1.0-1238]
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> ~[?:?]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> ~[?:?]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> ~[?:?]
> at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
