[
https://issues.apache.org/jira/browse/HIVE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brock Noland updated HIVE-4838:
-------------------------------
Attachment: HIVE-4838.patch
The rebased patch is on trunk. I was thinking about our plan and I have a
concern. Since we aren't allocating memory in large chunks when we do OOM it's
likely to be a very slow process with the local task doing lots of GC before
finally throwing an OOM. Therefore in the case where we fail with an OOM I
think it could be a significant negatively impact on performance. How about we
commit the patch as-is and then file a follow-on JIRA so that I or someone else
can prove or disprove this theory.
> Refactor MapJoin HashMap code to improve testability and readability
> --------------------------------------------------------------------
>
> Key: HIVE-4838
> URL: https://issues.apache.org/jira/browse/HIVE-4838
> Project: Hive
> Issue Type: Bug
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: HIVE-4838.patch, HIVE-4838.patch, HIVE-4838.patch,
> HIVE-4838.patch, HIVE-4838.patch, HIVE-4838.patch
>
>
> MapJoin is an essential component for high performance joins in Hive and the
> current code has done great service for many years. However, the code is
> showing it's age and currently suffers from the following issues:
> * Uses static state via the MapJoinMetaData class to pass serialization
> metadata to the Key, Row classes.
> * The api of a logical "Table Container" is not defined and therefore it's
> unclear what apis HashMapWrapper
> needs to publicize. Additionally HashMapWrapper has many used public methods.
> * HashMapWrapper contains logic to serialize, test memory bounds, and
> implement the table container. Ideally these logical units could be seperated
> * HashTableSinkObjectCtx has unused fields and unused methods
> * CommonJoinOperator and children use ArrayList on left hand side when only
> List is required
> * There are unused classes MRU, DCLLItemm and classes which duplicate
> functionality MapJoinSingleKey and MapJoinDoubleKeys
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
