Riddhish1 commented on issue #2101: URL: https://github.com/apache/stormcrawler/issues/2101#issuecomment-5683194535
## Performance Evaluation I evaluated the performance impact of removing the shared cache using a benchmark with **10 concurrent threads**, reused source URL objects, and domain filtering enabled. * **Cached:** 17,447,764 ops — **872,134 ops/sec** * **Uncached:** 16,193,576 ops — **809,506 ops/sec** * Cached implementation was **~7.7% faster** in the profiled run. * `PaidLevelDomain.getPLD()` accounted for **24.98%** of cached samples vs **33.24%** without caching. * Shorter runs were noisier, averaging roughly **14% higher throughput** with caching. These results indicate a measurable performance cost from removing the cache, though the exact improvement may vary by workload. ### Profiling Cached: [Cached profiling](https://github.com/user-attachments/assets/06cfba24-2b1c-4dbe-bb2f-9de1be0f1ce3) Uncached: [Uncached profiling](https://github.com/user-attachments/assets/306c40fc-c64c-4f38-aa01-a095fddf264a) Are we comfortable accepting this performance trade-off, given the concurrency benefits of removing the shared cache? @rzo1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
