Deneche A. Hakim created DRILL-4112:
---------------------------------------
Summary: BaseDataValueVector.getBuffers(false) doesn't return it's
inner buffer which affects transfer of ownership between allocators in
ExternalSort
Key: DRILL-4112
URL: https://issues.apache.org/jira/browse/DRILL-4112
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Deneche A. Hakim
As part of fixing DRILL-2274, to avoid running out of memory in sort when it
spilled to disk we close the _copierAllocator_ after we receive the last batch
from upstream, this way it releases all it's allocated memory back to it's
parent allocator. The assumption is that the _copierAllocator_ doesn't own any
buffers once a _mergeAndSpill_ is done and we no longer need it if we are no
longer
spilling to disk.
This causes it's accountor to complain that some buffers weren't closed (when
assertions are enabled), which goes against the previous assumption.
The _copierAllocator_ is used to allocate many batches while spilling but only
one is kept in memory and the following method transfers it's ownership to the
operator's allocator:
{code}
private void takeOwnership(VectorAccessible batch) {
for (VectorWrapper<?> w : batch) {
DrillBuf[] bufs = w.getValueVector().getBuffers(false);
for (DrillBuf buf : bufs) {
if (buf.isRootBuffer()) {
oContext.getAllocator().takeOwnership(buf);
}
}
}
}
{code}
The problem comes from how _BaseDataValueVector.getBuffers(boolean clear)_ is
implemented:
{code}
@Override
public DrillBuf[] getBuffers(boolean clear) {
DrillBuf[] out;
if (getBufferSize() == 0) {
out = new DrillBuf[0];
} else {
out = new DrillBuf[]{data};
if (clear) {
data.readerIndex(0);
data.retain(1);
}
}
if (clear) {
clear();
}
return out;
}
{code}
If we don't write any value into the vector, and call _getBuffers(false)_, it
won't return it's allocated buffer but an empty array instead. Thus, the buffer
won't be transferred to the operator's allocator.
The following unit test will exposes the problem:
{code}
@Test
public void testTakeOwnership() throws Exception {
final MaterializedField field =
MaterializedField.create(SchemaPath.getSimplePath(""),
NullableVarCharHolder.TYPE);
final DrillConfig drillConfig = DrillConfig.create();
final BufferAllocator allocator = RootAllocatorFactory.newRoot(drillConfig);
final BufferAllocator childAllocator = allocator.getChildAllocator(null,
10000, 20000, false);
final NullableVarCharVector vector = new NullableVarCharVector(field,
childAllocator);
vector.allocateNew(1024 * 10, 1024);
DrillBuf[] buffers = vector.getBuffers(false);
for (final DrillBuf buffer : buffers) {
allocator.takeOwnership(buffer);
}
childAllocator.close();
allocator.close();
}
{code}
When closing _childAllocator_ we get the following exception (when assertions
are enabled):
{noformat}
java.lang.IllegalStateException: Attempted to close accountor with 3 buffer(s)
still allocated.
Total 1 allocation(s) of byte size(s): 4100, at stack location:
org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:256)
org.apache.drill.exec.vector.UInt4Vector.allocateBytes(UInt4Vector.java:209)
org.apache.drill.exec.vector.UInt4Vector.allocateNew(UInt4Vector.java:191)
org.apache.drill.exec.vector.VarCharVector.allocateNew(VarCharVector.java:386)
org.apache.drill.exec.vector.NullableVarCharVector.allocateNew(NullableVarCharVector.java:224)
org.apache.drill.exec.record.vector.TestValueVector.testTakeOwnership(TestValueVector.java:123)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:606)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:120)
mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:65)
mockit.integration.junit4.internal.MockFrameworkMethod.invokeExplosively(MockFrameworkMethod.java:29)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:606)
mockit.internal.util.MethodReflection.invokeWithCheckedThrows(MethodReflection.java:95)
mockit.internal.annotations.MockMethodBridge.callMock(MockMethodBridge.java:76)
mockit.internal.annotations.MockMethodBridge.invoke(MockMethodBridge.java:41)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Total 1 allocation(s) of byte size(s): 1024, at stack location:
org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:256)
org.apache.drill.exec.vector.UInt1Vector.allocateBytes(UInt1Vector.java:209)
org.apache.drill.exec.vector.UInt1Vector.allocateNew(UInt1Vector.java:191)
org.apache.drill.exec.vector.NullableVarCharVector.allocateNew(NullableVarCharVector.java:225)
org.apache.drill.exec.record.vector.TestValueVector.testTakeOwnership(TestValueVector.java:123)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:606)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:120)
mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:65)
mockit.integration.junit4.internal.MockFrameworkMethod.invokeExplosively(MockFrameworkMethod.java:29)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:606)
mockit.internal.util.MethodReflection.invokeWithCheckedThrows(MethodReflection.java:95)
mockit.internal.annotations.MockMethodBridge.callMock(MockMethodBridge.java:76)
mockit.internal.annotations.MockMethodBridge.invoke(MockMethodBridge.java:41)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Total 1 allocation(s) of byte size(s): 10240, at stack location:
org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:256)
org.apache.drill.exec.vector.VarCharVector.allocateNew(VarCharVector.java:385)
org.apache.drill.exec.vector.NullableVarCharVector.allocateNew(NullableVarCharVector.java:224)
org.apache.drill.exec.record.vector.TestValueVector.testTakeOwnership(TestValueVector.java:123)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:606)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:120)
mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:65)
mockit.integration.junit4.internal.MockFrameworkMethod.invokeExplosively(MockFrameworkMethod.java:29)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:606)
mockit.internal.util.MethodReflection.invokeWithCheckedThrows(MethodReflection.java:95)
mockit.internal.annotations.MockMethodBridge.callMock(MockMethodBridge.java:76)
mockit.internal.annotations.MockMethodBridge.invoke(MockMethodBridge.java:41)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
java.lang.IllegalStateException: Attempted to close accountor with 3 buffer(s)
still allocated.
Total 1 allocation(s) of byte size(s): 4100, at stack location:
org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:256)
org.apache.drill.exec.vector.UInt4Vector.allocateBytes(UInt4Vector.java:209)
org.apache.drill.exec.vector.UInt4Vector.allocateNew(UInt4Vector.java:191)
org.apache.drill.exec.vector.VarCharVector.allocateNew(VarCharVector.java:386)
org.apache.drill.exec.vector.NullableVarCharVector.allocateNew(NullableVarCharVector.java:224)
org.apache.drill.exec.record.vector.TestValueVector.testTakeOwnership(TestValueVector.java:123)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:120)
mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:65)
mockit.integration.junit4.internal.MockFrameworkMethod.invokeExplosively(MockFrameworkMethod.java:29)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
mockit.internal.util.MethodReflection.invokeWithCheckedThrows(MethodReflection.java:95)
mockit.internal.annotations.MockMethodBridge.callMock(MockMethodBridge.java:76)
mockit.internal.annotations.MockMethodBridge.invoke(MockMethodBridge.java:41)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Total 1 allocation(s) of byte size(s): 1024, at stack location:
org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:256)
org.apache.drill.exec.vector.UInt1Vector.allocateBytes(UInt1Vector.java:209)
org.apache.drill.exec.vector.UInt1Vector.allocateNew(UInt1Vector.java:191)
org.apache.drill.exec.vector.NullableVarCharVector.allocateNew(NullableVarCharVector.java:225)
org.apache.drill.exec.record.vector.TestValueVector.testTakeOwnership(TestValueVector.java:123)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:120)
mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:65)
mockit.integration.junit4.internal.MockFrameworkMethod.invokeExplosively(MockFrameworkMethod.java:29)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
mockit.internal.util.MethodReflection.invokeWithCheckedThrows(MethodReflection.java:95)
mockit.internal.annotations.MockMethodBridge.callMock(MockMethodBridge.java:76)
mockit.internal.annotations.MockMethodBridge.invoke(MockMethodBridge.java:41)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Total 1 allocation(s) of byte size(s): 10240, at stack location:
org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:256)
org.apache.drill.exec.vector.VarCharVector.allocateNew(VarCharVector.java:385)
org.apache.drill.exec.vector.NullableVarCharVector.allocateNew(NullableVarCharVector.java:224)
org.apache.drill.exec.record.vector.TestValueVector.testTakeOwnership(TestValueVector.java:123)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:120)
mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:65)
mockit.integration.junit4.internal.MockFrameworkMethod.invokeExplosively(MockFrameworkMethod.java:29)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
mockit.internal.util.MethodReflection.invokeWithCheckedThrows(MethodReflection.java:95)
mockit.internal.annotations.MockMethodBridge.callMock(MockMethodBridge.java:76)
mockit.internal.annotations.MockMethodBridge.invoke(MockMethodBridge.java:41)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
at org.apache.drill.exec.memory.Accountor.close(Accountor.java:381)
at
org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:327)
at
org.apache.drill.exec.record.vector.TestValueVector.testTakeOwnership(TestValueVector.java:130)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
