[
https://issues.apache.org/jira/browse/DRILL-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16327513#comment-16327513
]
ASF GitHub Bot commented on DRILL-6002:
---------------------------------------
Github user vrozov commented on a diff in the pull request:
https://github.com/apache/drill/pull/1058#discussion_r161842736
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/spill/SpillSet.java
---
@@ -104,9 +107,17 @@
* nodes provide insufficient local disk space)
*/
+ // The buffer size is calculated as LCM of the Hadoop internal
checksum buffer (9 * checksum length), where
--- End diff --
@paul-rogers Changed `TRANSFER_SIZE` to 72K. It is calculated as LCM
between Hadoop internal checksum buffer and MapR FS page size to make writes
aligned on internal buffer boundaries.
> Avoid memory copy from direct buffer to heap while spilling to local disk
> -------------------------------------------------------------------------
>
> Key: DRILL-6002
> URL: https://issues.apache.org/jira/browse/DRILL-6002
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Vlad Rozov
> Assignee: Vlad Rozov
> Priority: Major
>
> When spilling to a local disk or to any file system that supports
> WritableByteChannel it is preferable to avoid copy from off-heap to java heap
> as WritableByteChannel can work directly with the off-heap memory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
