[ https://issues.apache.org/jira/browse/HIVE-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prasanth Jayachandran updated HIVE-17243: ----------------------------------------- Summary: Replace Hive's JavaDataModel with Java Object Layout (was: Replace Hive's DataModel with Java Object Layout) > Replace Hive's JavaDataModel with Java Object Layout > ---------------------------------------------------- > > Key: HIVE-17243 > URL: https://issues.apache.org/jira/browse/HIVE-17243 > Project: Hive > Issue Type: Improvement > Affects Versions: 3.0.0 > Reporter: Prasanth Jayachandran > Assignee: Prasanth Jayachandran > > org.apache.hadoop.hive.ql.util.JavaDataModel is used for estimating memory of > objects in many places. The problem with this approach is that manual > accounting for all fields and references has to be done. This will also be > problematic for cases where shallow object size vs deep object sizes are > required. > Hash table memory monitoring does accounting for size of the hash tables. The > estimated sizes of hash tables are often very different for non-vector > operators whereas the estimates are close to actual object size for > vectorized operators. Also addition of fields requires manual changes to > memory monitoring. > Java object layout is openjdk project that can provide shallow and deep > object sizes without using java agents. We can leverage that for much more > accurate memory estimates. -- This message was sent by Atlassian JIRA (v6.4.14#64029)