[
https://issues.apache.org/jira/browse/RANGER-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duong updated RANGER-4251:
--------------------------
Description:
The Ranger plugin for Ozone was implemented as per RANGER-2325.
When running performance testing on Ozone, we notice that the Ranger plugin
contributes significantly to Ozone latency and it gets worse when Ozone is
under heavy load, i.e. 100K OPPS.
!ozone-ranger-flamegraph.png|width=887,height=265!
This ticket tracks the effort to improve the performance of the Ranger plugin
for Ozone.
Source of inefficiencies identified so far:
# Ranger is logging/auditing all the requests, including reads that are
accepted. This is quite expensive and involves lock contention (internally in
LinkedBlockingQueue) and thus does not scale well for hundred thousands TPS. It
will be addressed by RANGER-4252.
# RangerPolicyEngineImpl spends most of its time looking for matching
evaluators for requests, aka
"policyRepository.getLikelyMatchPolicyEvaluators(request, policyType);". This
contains list creation, filter, and sorting so is not cheap. We should check if
it's catchable or optimizable.
was:
The Ranger plugin for Ozone was implemented as per RANGER-2325.
When running performance testing on Ozone, we notice that the Ranger plugin
contributes significantly to Ozone latency and it gets worse when Ozone is
under heavy load, i.e. 100K OPPS.
!ozone-ranger-flamegraph.png|width=887,height=265!
This ticket tracks the effort to improve the performance of the Ranger plugin
for Ozone.
Source of inefficiencies identified so far:
# Ranger is logging/auditing all the requests, including reads that are
accepted. This is too expensive and will be addressed by RANGER-4252.
# RangerPolicyEngineImpl spends most of its time looking for matching
evaluators, aka "policyRepository.getLikelyMatchPolicyEvaluators(request,
policyType);". This seems to be costly and
> Performance improvement for Ozone plugin
> ----------------------------------------
>
> Key: RANGER-4251
> URL: https://issues.apache.org/jira/browse/RANGER-4251
> Project: Ranger
> Issue Type: Improvement
> Components: plugins
> Reporter: Duong
> Priority: Major
> Attachments: image-2023-05-24-15-14-33-403.png,
> ozone-load-test-flamegraph.html, ozone-ranger-flamegraph.png
>
>
> The Ranger plugin for Ozone was implemented as per RANGER-2325.
> When running performance testing on Ozone, we notice that the Ranger plugin
> contributes significantly to Ozone latency and it gets worse when Ozone is
> under heavy load, i.e. 100K OPPS.
> !ozone-ranger-flamegraph.png|width=887,height=265!
> This ticket tracks the effort to improve the performance of the Ranger plugin
> for Ozone.
> Source of inefficiencies identified so far:
> # Ranger is logging/auditing all the requests, including reads that are
> accepted. This is quite expensive and involves lock contention (internally in
> LinkedBlockingQueue) and thus does not scale well for hundred thousands TPS.
> It will be addressed by RANGER-4252.
> # RangerPolicyEngineImpl spends most of its time looking for matching
> evaluators for requests, aka
> "policyRepository.getLikelyMatchPolicyEvaluators(request, policyType);". This
> contains list creation, filter, and sorting so is not cheap. We should check
> if it's catchable or optimizable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
