Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1234#discussion_r183611338
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/exec/store/bson/TestBsonRecordReader.java
---
@@ -49,17 +50,20 @@
import org.junit.BeforeClass;
import org.junit.Test;
-public class TestBsonRecordReader extends BaseTestQuery {
+public class TestBsonRecordReader {
+ private static BufferAllocator allocator;
private static VectorContainerWriter writer;
private static TestOutputMutator mutator;
+ private static DrillBuf buffer;
private static BsonRecordReader bsonReader;
@BeforeClass
public static void setUp() {
- BufferAllocator bufferAllocator = getDrillbitContext().getAllocator();
- mutator = new TestOutputMutator(bufferAllocator);
+ allocator = new RootAllocator(100_000_000);
--- End diff --
After testing just now, the lowest I could push this was 9,000,000 bytes.
Going to 8,000,000 bytes causes an OOM. It looks like the BsonRecordReader
allocates a bunch of value vectors. I will adjust this to 9,000,000.
```
org.apache.drill.exec.exception.OutOfMemoryException: Unable to allocate
buffer of size 8192 due to memory limit (800000). Current allocation: 795648
at
org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:236)
at
org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:211)
at
org.apache.drill.exec.vector.UInt1Vector.reallocRaw(UInt1Vector.java:262)
at
org.apache.drill.exec.vector.UInt1Vector.reAlloc(UInt1Vector.java:251)
at
org.apache.drill.exec.vector.UInt1Vector$Mutator.setSafe(UInt1Vector.java:480)
at
org.apache.drill.exec.vector.NullableVarCharVector$Mutator.setSafe(NullableVarCharVector.java:608)
at
org.apache.drill.exec.vector.complex.impl.NullableVarCharWriterImpl.write(NullableVarCharWriterImpl.java:110)
at
org.apache.drill.exec.store.bson.BsonRecordReader.writeString(BsonRecordReader.java:276)
at
org.apache.drill.exec.store.bson.BsonRecordReader.writeBinary(BsonRecordReader.java:205)
at
org.apache.drill.exec.store.bson.BsonRecordReader.writeToListOrMap(BsonRecordReader.java:117)
at
org.apache.drill.exec.store.bson.BsonRecordReader.write(BsonRecordReader.java:75)
at
org.apache.drill.exec.store.bson.TestBsonRecordReader.testBinaryTypes(TestBsonRecordReader.java:244)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
```
---
