-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74090/
-----------------------------------------------------------
(Updated Aug. 17, 2022, 4:54 a.m.)
Review request for ranger, bhavik patel, Dhaval Shah, Abhay Kulkarni, Madhan
Neethiraj, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
Changes
-------
Fixed issues. Removed unimplemented methods such as clear(). Used python
argparse library. Added Licenses. Improved naming conventions. Used inheritance
for service store objects.
Bugs: RANGER-3852
https://issues.apache.org/jira/browse/RANGER-3852
Repository: ranger
Description
-------
Simulation of user scenarios of high load or long duration of requests required
to analyze performance of the system and detect abnormalities in case of high
latency, memory, cpu usage etc
Diffs (updated)
-----
ranger-tools/src/main/python/README.md PRE-CREATION
ranger-tools/src/main/python/config/README.md PRE-CREATION
ranger-tools/src/main/python/outputs/README.md PRE-CREATION
ranger-tools/src/main/python/performance_analyzer.py PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/__init__.py PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/perf_globals.py
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_assets/__init__.py
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_assets/default_primary_config.json
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_assets/default_secondary_config.json
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_assets/path.py
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_object_stores/__init__.py
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_object_stores/base_object_stores.py
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_object_stores/random_generators.py
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_object_stores/service_object_stores.py
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_utils/__init__.py
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_utils/config_utils.py
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_utils/dataframe_utils.py
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_utils/extra_unused_utils.py
PRE-CREATION
ranger-tools/src/main/python/ranger_performance_tool/ranger_perf_utils/logging_utils.py
PRE-CREATION
ranger-tools/src/main/python/requirements.txt PRE-CREATION
ranger-tools/src/main/python/setup_performance_analyzer.py PRE-CREATION
Diff: https://reviews.apache.org/r/74090/diff/2/
Changes: https://reviews.apache.org/r/74090/diff/1-2/
Testing
-------
Aim:
The tool has been developed for developers. 2 config files (primary and
secondary) have been provided which the user can modify for their ranger
environment including different api's and the number of calls for each api that
need to be tested. Both API metrics (such as latency) as well as system metrics
(such as CPU usage, memory usage etc) are obtained after the tool is run. The
tool provides a framework with a lot of reusable components which can be used
to extend the functionality to other API's which are not yet supported.
Documentation and steps to run the tool:
README.md file at the root level (ranger-tools/src/main/python directory) can
be used as reference for the steps that need to be followed to run the tool.
The tool has been embedded with python docstrings which can be used to generate
full documentation of the developed tool (details can be found in the README
file).
Currently supported API's for the tool:
["create_policy", "update_policy_by_id", "get_policy_by_id",
"delete_policy_by_id"]
Sanity checks and testing:
The steps in the README.md file were followed in 2 separate instances
independently to verify the correctness of the setup steps provided and sanity
test of the tool developed.
The tool has been tested with multiple number of calls and different api's
ranging from 1-50 for different api's
File Attachments (updated)
----------------
RANGER-3852-performance-and-scalability-analyzer-to.patch
https://reviews.apache.org/media/uploaded/files/2022/08/17/f750dde5-c39b-44b0-b293-ddf1e372c658__RANGER-3852-performance-and-scalability-analyzer-to.patch
RANGER-3852-performance-and-scalability-analyzer-to.patch
https://reviews.apache.org/media/uploaded/files/2022/08/17/4a3e6441-1c60-4172-95a0-77d7b098deab__RANGER-3852-performance-and-scalability-analyzer-to.patch
Thanks,
Fateh Singh
