[
https://issues.apache.org/jira/browse/DRILL-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275244#comment-16275244
]
ASF GitHub Bot commented on DRILL-6002:
---------------------------------------
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1058#discussion_r154477971
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/managed/BatchGroup.java
---
@@ -280,17 +278,14 @@ private void closeInputStream() throws IOException {
logger.trace("Summary: Read {} bytes from {}", readLength, path);
}
- public long closeOutputStream() throws IOException {
- if (outputStream == null) {
- return 0;
+ public void closeWriter() throws IOException {
+ if (writer == null) {
+ return;
}
- long writeSize = spillSet.getPosition(outputStream);
- spillSet.tallyWriteBytes(writeSize);
- outputStream.close();
- outputStream = null;
+ long writeSize = writer.close();
+ long timeNs = writer.timeNs();
--- End diff --
See comment above. Updating spill set here decouples the writer from the
spill set. The `BatchGroup` already knows about the spill set.
> Avoid memory copy from direct buffer to heap while spilling to local disk
> -------------------------------------------------------------------------
>
> Key: DRILL-6002
> URL: https://issues.apache.org/jira/browse/DRILL-6002
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Vlad Rozov
> Assignee: Vlad Rozov
>
> When spilling to a local disk or to any file system that supports
> WritableByteChannel it is preferable to avoid copy from off-heap to java heap
> as WritableByteChannel can work directly with the off-heap memory.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
