I actually don't get your concern... looking at the TimedBuffer (after
your changes), there still a check about the timeout:
final boolean checkpoint = System.nanoTime() > lastFlushTime + timeout;
if (checkpoint || currentPendingSyncs >= estimatedOptimalBatch) {
That is, we won't flush the write unless the timeout is achieved...
that means the timeout calculated during the server.create (which is
using SyncCalculation) is still used... And this *should* work as the
IO limiter.
What am I missing here?
On Wed, May 10, 2017 at 1:13 PM, Clebert Suconic
<[email protected]> wrote:
> The TimedBuffer actually had this functionality. I didn't realize this
> semantic change on the timed waits. I'm strongly thinking about
> reverting to the previous functionality.
>
> This could even be the case of -1 the current releases, as this will
> just keep writing bursts beyond the disk capacity.
>
> That's why we calculate the IOPs the disk can perform when we create a
> journal, but the TimedBuffer is just ignoring such setting after your
> changes.
>
> On Wed, May 10, 2017 at 4:30 AM, nigro_franz <[email protected]> wrote:
>> Hi guys!
>>
>> After the changes to TimedBuffer to make the Journal capable to adapt to the
>> current load, I've had instructive feedbacks/advices about IOPS limits that
>> could be reached on too high and long burst (depending on disk capacity).
>>
>> There is the need to separate cleanly the concerns related to disk capacity
>> and load requested of the system: a configurable IOLimiter could help on it.
>>
>> A first attemp to do it is in this branch:
>> https://github.com/franz1981/activemq-artemis/blob/iops_limiter
>>
>> The possible configuration of such IOPS limiter could be:
>>
>> 1) no limiter (default): the disk naturally create backpressure on too high
>> loads
>> 2) balanced: "best effort" IO limiter based on IOPS computed by the broker
>> tuning tools (ie SyncCalculation)
>> https://github.com/franz1981/activemq-artemis/blob/iops_limiter/artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/BestEffortIOLimiter.java
>>
>> I was thinking to add a strict one too, but AFAIK strict limiters tends to
>> over-compensate the controlled systems, killing its latencies/throughput
>> when it is not be supposed to be done...just my 2 cents.
>>
>> wdyt?
>>
>> Franz
>>
>>
>>
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875.html
>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>
>
>
> --
> Clebert Suconic
--
Clebert Suconic
