[ https://issues.apache.org/jira/browse/HIVE-17754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mithun Radhakrishnan updated HIVE-17754: ---------------------------------------- Attachment: HIVE-17754.2-branch-2.patch > InputJobInfo in Pig UDFContext is heavyweight, and causes OOMs in Tez AMs > ------------------------------------------------------------------------- > > Key: HIVE-17754 > URL: https://issues.apache.org/jira/browse/HIVE-17754 > Project: Hive > Issue Type: Bug > Components: HCatalog > Affects Versions: 2.2.0, 3.0.0 > Reporter: Mithun Radhakrishnan > Assignee: Mithun Radhakrishnan > Attachments: HIVE-17754.1.patch, HIVE-17754.2-branch-2.patch, > HIVE-17754.2.patch > > > HIVE-9845 dealt with reducing the size of HCat split-info, to improve > job-launch times for Pig/HCat jobs. > For large Pig queries that scan a large number of Hive partitions, it was > found that the Pig {{UDFContext}} stored full-fat HCat {{InputJobInfo}} > objects, thus blowing out the Pig Tez AM. Since this information is already > stored in the {{HCatSplit}}, the serialization of {{InputJobInfo}} can be > spared. -- This message was sent by Atlassian JIRA (v6.4.14#64029)