[
https://issues.apache.org/jira/browse/ASTERIXDB-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244307#comment-16244307
]
Wenhai commented on ASTERIXDB-2156:
-----------------------------------
I should be mentioned that, in the current master we will encounter the first
error shown above where the database will corrupt.on 300 million twitter
datasets, regardless of the indexes (We try to reduce the index with only only
fullindex on twitter text, that error still exists.)
After we return the version in Oct. 22, the error disappears.
> Encounter error during feed, in both inverted index and rtree
> -------------------------------------------------------------
>
> Key: ASTERIXDB-2156
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2156
> Project: Apache AsterixDB
> Issue Type: Bug
> Environment: 4nc 8 partitions per node. Linux
> Reporter: Wenhai
>
> Hi, all
> When we try cloudberry recently, we encounter the following error when we use
> feed on 0.3 billions twitter records.
> We can reproduce the following errors
> {\noformat}
> SEVERE: Failed merge operation on {"class" : "LSMInvertedIndex", "dir" :
> "/kolla/asterixdb/asterix/node4/8/storage/storage/partition_31/twitter/ds_tweet_idx_text_idx",
> "memory" : 2, "disk" : 14}
> java.lang.Error: Maximum lock count exceeded
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:528)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:488)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1282)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
> at
> org.apache.hyracks.storage.common.buffercache.CachedPage.acquireReadLatch(CachedPage.java:120)
> at
> org.apache.hyracks.storage.am.btree.impls.BTree.acquireLatch(BTree.java:542)
> at
> org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:570)
> at
> org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:596)
> at
> org.apache.hyracks.storage.am.btree.impls.BTree.search(BTree.java:198)
> at
> org.apache.hyracks.storage.am.btree.impls.BTree.access$300(BTree.java:69)
> at
> org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.search(BTree.java:902)
> at
> org.apache.hyracks.storage.am.lsm.invertedindex.ondisk.OnDiskInvertedIndexRangeSearchCursor.open(OnDiskInvertedIndexRangeSearchCursor.java:74)
> at
> org.apache.hyracks.storage.am.lsm.invertedindex.ondisk.OnDiskInvertedIndex$OnDiskInvertedIndexAccessor.rangeSearch(OnDiskInvertedIndex.java:463)
> at
> org.apache.hyracks.storage.am.lsm.invertedindex.impls.LSMInvertedIndexRangeSearchCursor.open(LSMInvertedIndexRangeSearchCursor.java:68)
> at
> org.apache.hyracks.storage.am.lsm.invertedindex.impls.LSMInvertedIndex.search(LSMInvertedIndex.java:223)
> at
> org.apache.hyracks.storage.am.lsm.invertedindex.impls.LSMInvertedIndex.doMerge(LSMInvertedIndex.java:353)
> at
> org.apache.hyracks.storage.am.lsm.common.impls.AbstractLSMIndex.merge(AbstractLSMIndex.java:669)
> at
> org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.merge(LSMHarness.java:573)
> at
> org.apache.hyracks.storage.am.lsm.invertedindex.impls.LSMInvertedIndexAccessor.merge(LSMInvertedIndexAccessor.java:124)
> at
> org.apache.hyracks.storage.am.lsm.common.impls.MergeOperation.call(MergeOperation.java:45)
> at
> org.apache.hyracks.storage.am.lsm.common.impls.MergeOperation.call(MergeOperation.java:30)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {\noformat}
> or,
> {\noformat}
> Nov 07, 2017 3:39:55 PM
> org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness merge
> SEVERE: Failed merge operation on {"class" : "LSMRTreeWithAntiMatterTuples",
> "dir" :
> "/kolla/asterixdb/asterix/node4/1/storage/storage/partition_24/twitter/ds_tweet_idx_location_idx",
> "memory" : 2, "disk" : 19}
> org.apache.hyracks.algebricks.common.exceptions.NotImplementedException:
> Value provider for type missing is not implemented
> at
> org.apache.asterix.dataflow.data.nontagged.valueproviders.PrimitiveValueProviderFactory$1.getValue(PrimitiveValueProviderFactory.java:60)
> at
> org.apache.hyracks.storage.am.rtree.frames.RTreeNSMFrame.calculateMBRImpl(RTreeNSMFrame.java:131)
> at
> org.apache.hyracks.storage.am.rtree.frames.RTreeNSMFrame.adjustMBR(RTreeNSMFrame.java:152)
> at
> org.apache.hyracks.storage.am.rtree.impls.RTree$RTreeBulkLoader.propagateBulk(RTree.java:1047)
> at
> org.apache.hyracks.storage.am.rtree.impls.RTree$RTreeBulkLoader.add(RTree.java:948)
> at
> org.apache.hyracks.storage.am.lsm.common.impls.LSMIndexBulkLoader.add(LSMIndexBulkLoader.java:54)
> at
> org.apache.hyracks.storage.am.lsm.common.impls.ChainedLSMDiskComponentBulkLoader.add(ChainedLSMDiskComponentBulkLoader.java:54)
> at
> org.apache.hyracks.storage.am.lsm.rtree.impls.LSMRTreeWithAntiMatterTuples.doMerge(LSMRTreeWithAntiMatterTuples.java:195)
> at
> org.apache.hyracks.storage.am.lsm.common.impls.AbstractLSMIndex.merge(AbstractLSMIndex.java:667)
> at
> org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.merge(LSMHarness.java:574)
> at
> org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.merge(LSMTreeIndexAccessor.java:128)
> at
> org.apache.hyracks.storage.am.lsm.common.impls.MergeOperation.call(MergeOperation.java:45)
> at
> org.apache.hyracks.storage.am.lsm.common.impls.MergeOperation.call(MergeOperation.java:30)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {\noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
