optimize group by hash map memory
---------------------------------
Key: HIVE-1758
URL: https://issues.apache.org/jira/browse/HIVE-1758
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Namit Jain
Assignee: Siying Dong
Group By map side's hash map consumes a lot of memory, thereby decreasing its
effectiveness.
We can use some of the optimizations from map-join to reduce the memory
footprint:
class KeyWrapper {
int hashcode;
ArrayList<Object> keys;
// decide whether this is already in hashmap (keys in hashmap are deepcopied
// version, and we need to use 'currentKeyObjectInspector').
boolean copy = false;
1. Changes keys to Array
2. Optimize the scenario when keys is of a small size (1,2) etc
Let us start profiling it and take it from there
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
