[jira] [Commented] (HIVE-13303) spill to YARN directories, not tmp, when available
[ https://issues.apache.org/jira/browse/HIVE-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15217106#comment-15217106 ] Gopal V commented on HIVE-13303: LGTM - +1. The deleteOnExit() is a little iffy since the process is not likely to exit soon, but that's inherited. > spill to YARN directories, not tmp, when available > -- > > Key: HIVE-13303 > URL: https://issues.apache.org/jira/browse/HIVE-13303 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13303.patch > > > RowContainer::setupWriter, HybridHashTableContainer::spillPartition, > (KeyValueContainer|ObjectContainer)::setupOutput, > VectorMapJoinRowBytesContainer::setupOutputFileStreams create files in tmp. > Maybe some other code does it too, those are the ones I see on the execution > path. When there are multiple YARN output directories and multiple tasks > running on a machine, it's better to use the YARN directories. The only > question is cleanup. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13303) spill to YARN directories, not tmp, when available
[ https://issues.apache.org/jira/browse/HIVE-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15206881#comment-15206881 ] Sergey Shelukhin commented on HIVE-13303: - [~gopalv] [~sseth] can you please review? > spill to YARN directories, not tmp, when available > -- > > Key: HIVE-13303 > URL: https://issues.apache.org/jira/browse/HIVE-13303 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13303.patch > > > RowContainer::setupWriter, HybridHashTableContainer::spillPartition, > (KeyValueContainer|ObjectContainer)::setupOutput, > VectorMapJoinRowBytesContainer::setupOutputFileStreams create files in tmp. > Maybe some other code does it too, those are the ones I see on the execution > path. When there are multiple YARN output directories and multiple tasks > running on a machine, it's better to use the YARN directories. The only > question is cleanup. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13303) spill to YARN directories, not tmp, when available
[ https://issues.apache.org/jira/browse/HIVE-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15204898#comment-15204898 ] Sergey Shelukhin commented on HIVE-13303: - TestParseNegative failed due to conflict with /tmp/ UDF jar (unrelated). The rest are unrelated. > spill to YARN directories, not tmp, when available > -- > > Key: HIVE-13303 > URL: https://issues.apache.org/jira/browse/HIVE-13303 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13303.patch > > > RowContainer::setupWriter, HybridHashTableContainer::spillPartition, > (KeyValueContainer|ObjectContainer)::setupOutput, > VectorMapJoinRowBytesContainer::setupOutputFileStreams create files in tmp. > Maybe some other code does it too, those are the ones I see on the execution > path. When there are multiple YARN output directories and multiple tasks > running on a machine, it's better to use the YARN directories. The only > question is cleanup. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13303) spill to YARN directories, not tmp, when available
[ https://issues.apache.org/jira/browse/HIVE-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202308#comment-15202308 ] Sergey Shelukhin commented on HIVE-13303: - [~gopalv] fyi > spill to YARN directories, not tmp, when available > -- > > Key: HIVE-13303 > URL: https://issues.apache.org/jira/browse/HIVE-13303 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13303.patch > > > RowContainer::setupWriter, HybridHashTableContainer::spillPartition, > (KeyValueContainer|ObjectContainer)::setupOutput, > VectorMapJoinRowBytesContainer::setupOutputFileStreams create files in tmp. > Maybe some other code does it too, those are the ones I see on the execution > path. When there are multiple YARN output directories and multiple tasks > running on a machine, it's better to use the YARN directories. The only > question is cleanup. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
