vyommani commented on PR #770:
URL: https://github.com/apache/ranger/pull/770#issuecomment-3669655958
Below is the output of the benchmark that I had run, blease find below the
results.
==============================================================================================================
=== BULK EVALUATION PERFORMANCE BENCHMARK ===
==============================================================================================================
Configuration:
Policies: 50,000
Threads: 24
Batches per thread: 30,000
CPU cores: 12
==============================================================================================================
>>> Scenario: Lock-free Snapshot
Configuration: deltaEnabled=false, inPlaceUpdates=false
isPolicyEngineMutable=false
----------------------------------------------------------------------------------------------------
Batch Size | Total Requests | Duration (s) | Throughput | vs
Baseline | Memory | P95 Latency
----------------------------------------------------------------------------------------------------
1 | 720,000 | 0.349 | 2,061,681 r/s |
- | 41.1 MB | 0.01 ms
10 | 7,200,000 | 2.214 | 3,252,033 r/s |
- | 33.2 MB | 0.32 ms
100 | 72,000,000 | 19.145 | 3,760,747 r/s |
- | 33.7 MB | 1.34 ms
1,000 | 720,000,000 | 191.461 | 3,760,563 r/s |
- | 42.2 MB | 9.14 ms
5,000 | 3,600,000,000 | 962.866 | 3,738,838 r/s |
- | 33.0 MB | 42.23 ms
10,000 | 7,200,000,000 | 2003.881 | 3,593,028 r/s |
- | 33.0 MB | 87.71 ms
>>> Scenario: Legacy Locked
Configuration: deltaEnabled=true, inPlaceUpdates=true
isPolicyEngineMutable=true
----------------------------------------------------------------------------------------------------
Batch Size | Total Requests | Duration (s) | Throughput | vs
Baseline | Memory | P95 Latency
----------------------------------------------------------------------------------------------------
1 | 720,000 | 0.483 | 1,489,744 r/s |
0.72x | 33.4 MB | 0.02 ms
10 | 7,200,000 | 2.389 | 3,014,398 r/s |
0.93x | 33.2 MB | 0.37 ms
100 | 72,000,000 | 20.235 | 3,558,228 r/s |
0.95x | 33.2 MB | 1.50 ms
1,000 | 720,000,000 | 201.901 | 3,566,102 r/s |
0.95x | 33.0 MB | 9.85 ms
5,000 | 3,600,000,000 | 1018.552 | 3,534,429 r/s |
0.95x | 33.0 MB | 45.05 ms
10,000 | 7,200,000,000 | 2100.223 | 3,428,207 r/s |
0.95x | 33.0 MB | 91.15 ms
========================================================================================================================
Legend: = 2x or greater speedup | r/s = requests per second
========================================================================================================================
=== SCALABILITY TEST: Performance vs Policy Count ===
Configuration: batch=1000, threads=24, iterations=10000, runs=5
------------------------------------------------------------------------------------------------------------------------
Mode | Policy Count | Avg Duration (s) | Avg
Throughput | StdDev % | vs Locked
------------------------------------------------------------------------------------------------------------------------
Snapshot | 1,000 | 2.793 |
3,597,980 r/s | 6.63% | -
Locked | 1,000 | 2.907 |
3,454,039 r/s | 6.00% | -
→ Speedup | 1,000 | - |
- | - | 1.04x
Snapshot | 10,000 | 2.873 |
3,506,523 r/s | 8.35% | -
Locked | 10,000 | 2.903 |
3,455,737 r/s | 5.48% | -
→ Speedup | 10,000 | - |
- | - | 1.01x
Snapshot | 50,000 | 2.740 |
3,653,690 r/s | 3.30% | -
Locked | 50,000 | 2.884 |
3,478,907 r/s | 5.40% | -
→ Speedup | 50,000 | - |
- | - | 1.05x
Snapshot | 100,000 | 3.094 |
3,249,175 r/s | 7.74% | -
Locked | 100,000 | 2.948 |
3,406,937 r/s | 6.52% | -
→ Speedup | 100,000 | - |
- | - | 0.95x
------------------------------------------------------------------------------------------------------------------------
=== CORRECTNESS TEST: Snapshot vs Locked Evaluation ===
All 500 requests produced identical results
- Allowed: 0
- Denied: 500
=== CONCURRENCY STRESS TEST: Policy Updates During Evaluation ===
Starting 9 evaluation threads + 1 update thread...
All threads completed successfully
No race conditions or crashes detected
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
