[
https://issues.apache.org/jira/browse/RANGER-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Madhan Neethiraj resolved RANGER-4761.
--------------------------------------
Fix Version/s: 3.0.0
2.5.0
Resolution: Fixed
{noformat}
commit fa18ffb9e8195993f2c8f3289fbdffecafb2bd9a
Author: Fateh Singh <[email protected]>
Date: Mon May 6 13:15:49 2024 -0700
RANGER-4761: [hbase] make lazy memory allocation for family map (#307)
Map<String, Set> getColumnFamilies(Map<byte[], ? extends Collection<?>>
families) becomes a bottleneck in multiget and multiput workloads wherein
hundred/thousands of columns may be accessed together in a single request as it
is a computatio
nally heavy function converting bytes to string and type-casting Collection to
set of strings.
In the patch, make lazy memory allocation for family map instead of ahead
of time memory allocation for family map of type Map<String, Set>.
{noformat}
> Reduce memory footprint of hbase plugin
> ---------------------------------------
>
> Key: RANGER-4761
> URL: https://issues.apache.org/jira/browse/RANGER-4761
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
> Reporter: Fateh Singh
> Assignee: Fateh Singh
> Priority: Major
> Fix For: 3.0.0, 2.5.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> * Map<String, Set> getColumnFamilies(Map<byte[], ? extends Collection<?>>
> families) becomes a bottleneck in multiget and multiput workloads wherein
> hundred/thousands of columns may be accessed together in a single request as
> it is a computationally heavy function converting bytes to string and
> type-casting Collection to set of strings.
> The implementation needs to be revisited to reduce memory footprint
> make lazy memory allocation for family map instead of ahead of time memory
> allocation for family map of type Map<String, Set>. With this approach,
> memory footprint will get reduced even further when enabling column auth
> optimization supported by RANGER-4670
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
