[
https://issues.apache.org/jira/browse/DRILL-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16584782#comment-16584782
]
ASF GitHub Bot commented on DRILL-6644:
---------------------------------------
vvysotskyi commented on issue #1409: DRILL-6644: Don't reserve space for
incoming probe batches unnecessarily during the build phase.
URL: https://github.com/apache/drill/pull/1409#issuecomment-414060965
@ilooner, this PR causes Advanced tests failures:
```
tpcds/tpcds_sf1/original/parquet/query11.sql
tpcds/tpcds_sf1/original/parquet/query4.sql
java.sql.SQLException: SYSTEM ERROR: IllegalStateException
Fragment 35:1
[Error Id: 81d1f358-6871-4ab4-a79c-d632b6e5c1ae on cv1:31010]
(java.lang.IllegalStateException) null
com.google.common.base.Preconditions.checkState():158
org.apache.drill.exec.physical.impl.join.BatchSizePredictorImpl.predictBatchSize():76
org.apache.drill.exec.physical.impl.join.HashJoinMemoryCalculatorImpl$PostBuildCalculationsImpl.initialize():635
org.apache.drill.exec.physical.impl.join.HashJoinBatch.executeBuildPhase():844
org.apache.drill.exec.physical.impl.join.HashJoinBatch.innerNext():414
org.apache.drill.exec.physical.impl.join.HashJoinBatch.innerNext():544
org.apache.drill.exec.record.AbstractRecordBatch.next():172
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():142
org.apache.drill.exec.record.AbstractRecordBatch.next():172
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():142
org.apache.drill.exec.record.AbstractRecordBatch.next():172
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.test.generated.HashAggregatorGen75169.doWork():649
org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.innerNext():273
org.apache.drill.exec.record.AbstractRecordBatch.next():172
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():142
org.apache.drill.exec.record.AbstractRecordBatch.next():172
org.apache.drill.exec.physical.impl.BaseRootExec.next():103
org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext():93
org.apache.drill.exec.physical.impl.BaseRootExec.next():93
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():294
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():281
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():422
org.apache.hadoop.security.UserGroupInformation.doAs():1595
org.apache.drill.exec.work.fragment.FragmentExecutor.run():281
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1149
java.util.concurrent.ThreadPoolExecutor$Worker.run():624
java.lang.Thread.run():748
```
Could you please take a look?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> In Some Cases The HashJoin Memory Calculator Over Reserves Memory For The
> Probe Side During The Build Phase
> -----------------------------------------------------------------------------------------------------------
>
> Key: DRILL-6644
> URL: https://issues.apache.org/jira/browse/DRILL-6644
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.14.0
> Reporter: Timothy Farkas
> Assignee: Timothy Farkas
> Priority: Major
> Labels: ready-to-commit
> Fix For: 1.15.0
>
>
> There are two cases where the HashJoin Memory calculator over reserves memory:
> 1. It reserves a maximum incoming probe batch size during the build phase.
> This is not really necessary because we will not fetch probe data until the
> probe phase. We only have to account for the data received during
> OK_NEW_SCHEMA.
> 2. https://issues.apache.org/jira/browse/DRILL-6646
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
