[jira] [Commented] (HIVE-2368) Slow dropping of partitions caused by full listing of storage descriptors
[ https://issues.apache.org/jira/browse/HIVE-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13147389#comment-13147389 ] Hudson commented on HIVE-2368: -- Integrated in Hive-0.8.0-SNAPSHOT-h0.21 #89 (See [https://builds.apache.org/job/Hive-0.8.0-SNAPSHOT-h0.21/89/]) HIVE-2368. Slow dropping of partitions caused by full listing of storage descriptors (Sohan Jain via pauly) pauly : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1199915 Files : * /hive/branches/branch-0.8/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java Slow dropping of partitions caused by full listing of storage descriptors - Key: HIVE-2368 URL: https://issues.apache.org/jira/browse/HIVE-2368 Project: Hive Issue Type: Bug Components: Metastore Reporter: Sohan Jain Assignee: Sohan Jain Fix For: 0.9.0 Attachments: HIVE-2368.1.patch To determine if a column descriptor is unused, we call listStorageDescriptorsWithCD(), which may return a big list of SDs. This can severely slow down dropping partitions. We can add a maximum number of SDs to return, and just ask for 1 SD, since we are just doing an existential check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2368) Slow dropping of partitions caused by full listing of storage descriptors
[ https://issues.apache.org/jira/browse/HIVE-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13138065#comment-13138065 ] Paul Yang commented on HIVE-2368: - Backporting this to branch-0.8 as well. Slow dropping of partitions caused by full listing of storage descriptors - Key: HIVE-2368 URL: https://issues.apache.org/jira/browse/HIVE-2368 Project: Hive Issue Type: Bug Components: Metastore Reporter: Sohan Jain Assignee: Sohan Jain Fix For: 0.9.0 Attachments: HIVE-2368.1.patch To determine if a column descriptor is unused, we call listStorageDescriptorsWithCD(), which may return a big list of SDs. This can severely slow down dropping partitions. We can add a maximum number of SDs to return, and just ask for 1 SD, since we are just doing an existential check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2368) Slow dropping of partitions caused by full listing of storage descriptors
[ https://issues.apache.org/jira/browse/HIVE-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135387#comment-13135387 ] Paul Yang commented on HIVE-2368: - Committed. Thanks Sohan! Slow dropping of partitions caused by full listing of storage descriptors - Key: HIVE-2368 URL: https://issues.apache.org/jira/browse/HIVE-2368 Project: Hive Issue Type: Bug Components: Metastore Reporter: Sohan Jain Assignee: Sohan Jain Fix For: 0.9.0 Attachments: HIVE-2368.1.patch To determine if a column descriptor is unused, we call listStorageDescriptorsWithCD(), which may return a big list of SDs. This can severely slow down dropping partitions. We can add a maximum number of SDs to return, and just ask for 1 SD, since we are just doing an existential check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
