[
https://issues.apache.org/jira/browse/DRILL-5740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148168#comment-16148168
]
Paul Rogers commented on DRILL-5740:
------------------------------------
Thanks. Our original guess as to the cause appears to have been wrong. This may
be an issue in the Hash Agg code path since (knock on wood), we've not seen
something similar in the external sort tests. [~ben-zvi], can you take another
look?
> hash agg fail to read spill file
> --------------------------------
>
> Key: DRILL-5740
> URL: https://issues.apache.org/jira/browse/DRILL-5740
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
> Affects Versions: 1.12.0
> Reporter: Chun Chang
> Assignee: Paul Rogers
> Priority: Blocker
>
> -Build: | 1.12.0-SNAPSHOT | 11008d029bafa36279e3045c4ed1a64366080620
> -Multi-node drill cluster
> Running a query causing hash agg spill fails with the following error. And
> this seems to be a regression.
> {noformat}
> Execution Failures:
> /root/drill-test-framework/framework/resources/Advanced/hash-agg/spill/hagg5.q
> Query:
> select gby_date, gby_int32_rand, sum(int32_field), avg(float_field),
> min(boolean_field), count(double_rand) from
> dfs.`/drill/testdata/hagg/PARQUET-500M.parquet` group by gby_date,
> gby_int32_rand order by gby_date, gby_int32_rand limit 30
> Failed with exception
> java.sql.SQLException: SYSTEM ERROR: FileNotFoundException: File
> /tmp/drill/spill/10.10.30.168-31010/265f91f9-78d2-78a6-68ad-4709674efe0a_HashAgg_1-4-34/spill3
> does not exist
> Fragment 1:34
> [Error Id: 291a79f8-9b7a-485d-9404-e7b7fe1d8f1e on 10.10.30.168:31010]
> (java.lang.RuntimeException) java.io.FileNotFoundException: File
> /tmp/drill/spill/10.10.30.168-31010/265f91f9-78d2-78a6-68ad-4709674efe0a_HashAgg_1-4-34/spill3
> does not exist
>
> org.apache.drill.exec.physical.impl.aggregate.SpilledRecordbatch.<init>():67
>
> org.apache.drill.exec.test.generated.HashAggregatorGen1891.outputCurrentBatch():980
> org.apache.drill.exec.test.generated.HashAggregatorGen1891.doWork():617
> org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.innerNext():168
> org.apache.drill.exec.record.AbstractRecordBatch.next():164
> org.apache.drill.exec.record.AbstractRecordBatch.next():119
> org.apache.drill.exec.record.AbstractRecordBatch.next():109
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():133
> org.apache.drill.exec.record.AbstractRecordBatch.next():164
> org.apache.drill.exec.record.AbstractRecordBatch.next():119
> org.apache.drill.exec.record.AbstractRecordBatch.next():109
> org.apache.drill.exec.physical.impl.TopN.TopNBatch.innerNext():191
> org.apache.drill.exec.record.AbstractRecordBatch.next():164
> org.apache.drill.exec.record.AbstractRecordBatch.next():119
> org.apache.drill.exec.record.AbstractRecordBatch.next():109
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
>
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext():93
> org.apache.drill.exec.record.AbstractRecordBatch.next():164
> org.apache.drill.exec.physical.impl.BaseRootExec.next():105
>
> org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext():92
> org.apache.drill.exec.physical.impl.BaseRootExec.next():95
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():234
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():227
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():415
> org.apache.hadoop.security.UserGroupInformation.doAs():1595
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():227
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
