[jira] Commented: (HIVE-1828) show locks should not use getTable()/getPartition
[ https://issues.apache.org/jira/browse/HIVE-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12966981#action_12966981 ] Namit Jain commented on HIVE-1828: -- can you add the new patch ? also, can you add a negative test (if you have not done so already) ? show locks should not use getTable()/getPartition -- Key: HIVE-1828 URL: https://issues.apache.org/jira/browse/HIVE-1828 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: He Yongqiang Attachments: HIVE-1828.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1828) show locks should not use getTable()/getPartition
[ https://issues.apache.org/jira/browse/HIVE-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1828: - Status: Open (was: Patch Available) show locks should not use getTable()/getPartition -- Key: HIVE-1828 URL: https://issues.apache.org/jira/browse/HIVE-1828 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: He Yongqiang Attachments: HIVE-1828.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1830) mappers in group followed by joins may die OOM
mappers in group followed by joins may die OOM -- Key: HIVE-1830 URL: https://issues.apache.org/jira/browse/HIVE-1830 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Liyin Tang -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1830) mappers in group followed by joins may die OOM
[ https://issues.apache.org/jira/browse/HIVE-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12967105#action_12967105 ] Namit Jain commented on HIVE-1830: -- After HIVE-1642, joins are automatically converted into map-joins at physical optimization time. However, this may lead to problems. For eg: consider the query: select T1.val, count(1) from T1 join T2 on T1.key=T2.key group by T1.val This will have 2 map-reduce jobs, one for the join and the other for group by. Before HIVE-1642, the partial group for aggregation will be performed in the reducer where the join is performed. However, after HIVE-1642, the same will be performed in the mapper. The local task will confirm that there is just enough memory to hold the map-join data. Hoever, it does not take into account the memory needed for partial group by. So, in case there is group by followed by join, it is a good idea to reduce the memory given to the local task to validate if there is enough memory to fit small table - it can be controlled by a new configuration paramter, but it can be some default: say 70% of total memory (instead of 90%). Also, the group by may still run out of memory, so it might be a good idea to check in group by for free memory and periodically flush memory mappers in group followed by joins may die OOM -- Key: HIVE-1830 URL: https://issues.apache.org/jira/browse/HIVE-1830 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Liyin Tang -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1831) Add a option to run task to check map-join possibility in non-local mode
Add a option to run task to check map-join possibility in non-local mode Key: HIVE-1831 URL: https://issues.apache.org/jira/browse/HIVE-1831 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Liyin Tang In HIVE-1642, we run a local task to figure out if the small table can be held in memory, and then convert the join into a map-join. However, this can be a good idea for thin clients (which may not have enough memory). This should be made configurable - where the default can still be to run the task locally on the client machine, but an option should be added for thin clients, where the task would be run as a map-only task -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
