Deneche A. Hakim created DRILL-4542:
---------------------------------------
Summary: if external sort fails to spill to disk, memory is leaked
and wrong error message is displayed
Key: DRILL-4542
URL: https://issues.apache.org/jira/browse/DRILL-4542
Project: Apache Drill
Issue Type: Bug
Components: Execution - Relational Operators
Affects Versions: 1.6.0
Environment: When external sort fails spilling to disk, for example
because there is no space left on disk, the query fails with an error like this:
{noformat}
SYSTEM ERROR: IllegalStateException: Memory was leaked by query. Memory leaked:
(376832)
Allocator(op:1:0:6:ExternalSort) 20000000/376832/408432256/429496729
(res/actual/peak/limit)
{noformat}
2 problems here:
1. we are leaking memory
2. the "real" problem is not shown in the error message
Looking at the logs we can find the real issue, generally right after the
previous error (edited for the sake of clarity):
{noformat}
2016-03-26 02:22:28,576 [drill-executor-119] ERROR
o.a.d.exec.server.BootStrapContext -
org.apache.drill.exec.work.WorkManager$WorkerBee$2.run() leaked an exception.
org.apache.hadoop.fs.FSError: java.io.IOException: No space left on device
at
org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:249)
~[hadoop-common-2.7.0-mapr-1602.jar:na]
[...]
at
org.apache.drill.exec.cache.VectorAccessibleSerializable.writeToStream(VectorAccessibleSerializable.java:172)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.BatchGroup.addBatch(BatchGroup.java:96)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.mergeAndSpill(ExternalSortBatch.java:562)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:394)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
[...]
{noformat}
Reporter: Deneche A. Hakim
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
