[
https://issues.apache.org/jira/browse/HIVE-20509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618701#comment-16618701
]
Barnabas Maidics commented on HIVE-20509:
-----------------------------------------
[~gopalv]: I implemented a way, that doesn't create a new ArrayList for every
identical alias list. If two paths have the same aliases, they will use the
same ArrayList (If I understood you well, this is what you've thought of).
But there is a problem: When I used the same ArrayList object for every path,
the result was incorrect. Probably somewhere we change these ArrayList and if
we use the same one for every path, this will affect the whole pathToAliases
Map.
So as I understand, this would be a bigger change. I'll try to find where we
change these ArrayLists and why.
> Plan: fix wasted memory in plans with large partition counts
> ------------------------------------------------------------
>
> Key: HIVE-20509
> URL: https://issues.apache.org/jira/browse/HIVE-20509
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
> Reporter: Gopal V
> Assignee: Barnabas Maidics
> Priority: Minor
> Labels: newbie
> Attachments: HIVE-20509.2.patch, HIVE-20509.patch, after.png,
> before.png
>
>
> {code}
> public void addPathToAlias(Path path, String newAlias){
> ArrayList<String> aliases = pathToAliases.get(path);
> if (aliases == null) {
> aliases = new ArrayList<>();
> StringInternUtils.internUriStringsInPath(path);
> pathToAliases.put(path, aliases);
> }
> aliases.add(newAlias.intern());
> }
> {code}
> ArrayList::DEFAULT_CAPACITY is 10, so this wastes 500 bytes of memory due to
> the {{new ArrayList<>();}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
