Max Buffer Size Reached

[John Omernik]

Any thoughts on how to troubleshoot this (I have some fat json data going
into the buffers apparently) It's not huge data, just wide/complex (total
size is 1.4 GB)  Any thoughts on how to troubleshoot or settings I can use
to work through these errors?


Thanks!


John




Error: SYSTEM ERROR: OversizedAllocationException: Unable to expand the
buffer. Max allowed buffer size is reached.



Fragment 1:11



[Error Id: db21dea0-ddd7-4fcf-9fea-b5031e358dad on node1



  (org.apache.drill.exec.exception.OversizedAllocationException) Unable to
expand the buffer. Max allowed buffer size is reached.

    org.apache.drill.exec.vector.UInt1Vector.reAlloc():214

    org.apache.drill.exec.vector.UInt1Vector$Mutator.setValueCount():469


org.apache.drill.exec.vector.complex.ListVector$Mutator.setValueCount():324

    org.apache.drill.exec.physical.impl.ScanBatch.next():247

    org.apache.drill.exec.record.AbstractRecordBatch.next():119

    org.apache.drill.exec.record.AbstractRecordBatch.next():109

    org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51


org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():132

    org.apache.drill.exec.record.AbstractRecordBatch.next():162

    org.apache.drill.exec.record.AbstractRecordBatch.next():119


org.apache.drill.exec.test.generated.StreamingAggregatorGen1931.doWork():172


org.apache.drill.exec.physical.impl.aggregate.StreamingAggBatch.innerNext():167

    org.apache.drill.exec.record.AbstractRecordBatch.next():162

    org.apache.drill.exec.physical.impl.BaseRootExec.next():104


org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext():93

    org.apache.drill.exec.physical.impl.BaseRootExec.next():94

    org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():256

    org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():250

    java.security.AccessController.doPrivileged():-2

    javax.security.auth.Subject.doAs():415

    org.apache.hadoop.security.UserGroupInformation.doAs():1595

    org.apache.drill.exec.work.fragment.FragmentExecutor.run():250

    org.apache.drill.common.SelfCleaningRunnable.run():38

    java.util.concurrent.ThreadPoolExecutor.runWorker():1145

    java.util.concurrent.ThreadPoolExecutor$Worker.run():615

    java.lang.Thread.run():745 (state=,code=0)