[
https://issues.apache.org/jira/browse/ASTERIXDB-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15484913#comment-15484913
]
Wail Alkowaileet commented on ASTERIXDB-1641:
---------------------------------------------
If you look at
{noformat}org.apache.hyracks.control.common.dataset.ResultStateSweeper{noformat}
You will find that the result sweeper will destroy the result despite the fact
that the job is still running. It all depends on resultTTL (not
resultSweepThreshold. My bad).
In the documentation, it says that it only for async jobs, but I don't think
that's 100% true.
To reproduce:
1- Add breakpoint at line 64 in
{noformat}org.apache.hyracks.algebricks.runtime.serializer.ResultSerializerFactoryProvider{noformat}
2- Add breakpoint at line 75 in
{noformat}org.apache.hyracks.control.common.dataset.ResultStateSweeper{noformat}
3- Run any query you like.
4- From (1), execute line 64.
6- From (2), execute.
7- From (1), execute.
*Repeat one more time if that didn't produce an error.
You will see that your result file got deleted before writing.
> Hit HYR002 exception when query a large dataset
> -----------------------------------------------
>
> Key: ASTERIXDB-1641
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1641
> Project: Apache AsterixDB
> Issue Type: Bug
> Components: Storage
> Environment: Master
> Reporter: Jianfeng Jia
> Assignee: Ian Maxon
> Priority: Blocker
>
> The twitter dataset now is above 100G. I hit an error when send a simple
> query to the dataset:
> {code}
> SEVERE: Job failed on account of:
> HYR0002: org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException: null
> org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
> HYR0002: org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException: null
> at
> org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:212)
> at
> org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: HYR0002:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDat
> aException: null
> at
> org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
> at org.apache.hyracks.control.nc.Task.run(Task.java:319)
> ... 3 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataExceptio
> n: null
> at
> org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:218)
> at
> org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:83)
> at org.apache.hyracks.control.nc.Task.run(Task.java:263)
> ... 3 more
> Caused by: java.util.concurrent.ExecutionException:
> org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException: null
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at
> org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:212)
> ... 5 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
> org.apache.hyracks.api.exceptions.HyracksDataException: null
> at
> org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.nextFrame(IndexSearchOperatorNodePushable.java:187)
> at
> org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
> at
> org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.flushAndReset(AbstractOneInputOneOutputOneFramePushRuntime.java:63)
> at
> org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.flushIfNotFailed(AbstractOneInputOneOutputOneFramePushRuntime.java:
> 69)
> at
> org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.close(AbstractOneInputOneOutputOneFramePushRuntime.java:55)
> at
> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.close(AssignRuntimeFactory.java:125)
> at
> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.close(AlgebricksMetaOperatorDescriptor.java:153)
> at
> org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger.process(AbstractExternalSortRunMerger.java:167)
> at
> org.apache.hyracks.dataflow.std.sort.AbstractSorterOperatorDescriptor$MergeActivity$1.initialize(AbstractSorterOperatorDescriptor.java:194)
> at
> org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$0(SuperActivityOperatorNodePushable.java:83)
> at
> org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$1/1498483095.runAction(Unknown
> Source)
> at
> org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$1.call(SuperActivityOperatorNodePushable.java:205)
> at
> org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$1.call(SuperActivityOperatorNodePushable.java:202)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> Caused by: org.apache.hyracks.storage.am.btree.exceptions.BTreeException:
> org.apache.hyracks.api.exceptions.HyracksDataException: null
> at
> org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:827)
> at
> org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:673)
> at
> org.apache.hyracks.storage.am.btree.impls.BTree.search(BTree.java:204)
> at
> org.apache.hyracks.storage.am.btree.impls.BTree.access$600(BTree.java:74)
> at
> org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.search(BTree.java:966)
> at
> org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreePointSearchCursor.hasNext(LSMBTreePointSearchCursor.java:74)
> at
> org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeSearchCursor.hasNext(LSMBTreeSearchCursor.java:86)
> at
> org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.writeSearchResults(IndexSearchOperatorNodePushable.java:149)
> at
> org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.nextFrame(IndexSearchOperatorNodePushable.java:184)
> ... 16 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: null
> at
> org.apache.hyracks.control.nc.io.IOManager.syncRead(IOManager.java:175)
> at
> org.apache.hyracks.storage.common.buffercache.BufferCache.read(BufferCache.java:535)
> at
> org.apache.hyracks.storage.common.buffercache.BufferCache.pin(BufferCache.java:211)
> at
> org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:642)
> at
> org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:673)
> ... 24 more
> Caused by: java.nio.channels.ClosedChannelException
> at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:94)
> at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:673)
> at
> org.apache.hyracks.control.nc.io.IOManager.syncRead(IOManager.java:163)
> ... 28 more
> {code}
> [~imaxon] you could test it on actinium machine with the following query
> {code}
> count(for $t in dataset twitter.ds_tweet
> where $t.'create_at' >= datetime('2016-09-07T23:09:10.000Z') and
> $t.'create_at' < datetime('2016-09-09T23:09:10.000Z')
>
> return $t)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
