I've put the first attempt of relaxed/best effort IOPS limiter that works both for ASYNCIO and NIO too.
(The second commit on https://github.com/franz1981/activemq-artemis/tree/iops_limiter) The 2 main differences with the original one embedded in the TimedBuffer are: 1) it performs adapting compensation instead of upfront limitation, using the current IOPS perceived 2) it performs compensation against ALL flush operations like the forced ones due to a filled batch buffer, not handled in the original one In practical terms it doesn't overcompensate when the system is not really under stress, but It doesn't let the disk run out of IOPS due to forced batch buffer flushes. I've added in that experimental branch a real-time HdrHistogram instrumentation to record the distributions of: - flushed batch size in bytes (to understand the effectiveness of the coalescing heuristic) - "perceived" flush latency - compensations done by the limiter After some measurements, I've noticed that NIO and ASYNCIO behave very differently. ASYNCIO with maxIO>1 performs async flushes (histograms can't lie), while NIO si always synchronous and blocking, making the compensations more effective. Further investigations could will the best strategy to do it. If you have any idea, curiosity and/or advices about it please share it :) Franz -- View this message in context: http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726037.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
