Correct, :) seems like tester error. Apologies Sent from my iPhone
> On 12 May 2017, at 21:24, nigro_franz <[email protected]> wrote: > > Hence the new TimedBuffer didn't change drastically the latencies? > I've got some graph after the stress test with a maxed IO, if you're > curious :) > > 2017-05-12 22:19 GMT+02:00 Michael André Pearce [via ActiveMQ] < > [email protected]>: > >> Yup deleting the data on version change removes the issue. >> >> Sent from my iPhone >> >>> On 12 May 2017, at 20:54, Clebert Suconic <[hidden email] >> <http:///user/SendEmail.jtp?type=node&node=4726097&i=0>> wrote: >>> >>> Can you cleanup your data (as a test only?) rm -rf data. >>> >>> >>> >>> >>> perhaps the spikes you have is the compacting running? >>> >>> On Fri, May 12, 2017 at 3:51 PM, Michael André Pearce >>> <[hidden email] <http:///user/SendEmail.jtp?type=node&node=4726097&i=1>> >> wrote: >>>> All I'm doing is changing Artemis.profile of the instance to the >> different versions. >>>> >>>> >>>> Sent from my iPhone >>>> >>>>> On 12 May 2017, at 20:35, nigro_franz <[hidden email] >> <http:///user/SendEmail.jtp?type=node&node=4726097&i=2>> wrote: >>>>> >>>>> Too strange...have you tried with NIO too?It uses always the same >> buffer to >>>>> batch writes AFAIK.... >>>>> >>>>> 2017-05-12 21:30 GMT+02:00 Michael André Pearce [via ActiveMQ] < >>>>> [hidden email] <http:///user/SendEmail.jtp?type=node&node=4726097&i=3>>: >> >>>>> >>>>>> Ok so it isn't the times buffer changes. Just deployed 2.1.0 with and >>>>>> without the timed buffer changes. Running the same tests no >> difference. >>>>>> >>>>>> Must be something else. Causing the behaviour change. >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>>> On 12 May 2017, at 18:10, Michael André Pearce <[hidden email] >>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=0>> wrote: >>>>>>> >>>>>>> I've left for the day now. If I get time over the weekend I'll try >> see >>>>>> if I can make a build of 2.1.0 without that change and see if it >> makes any >>>>>> difference. >>>>>>> >>>>>>> Can someone point me to the PR for that change, so I know what I'm >>>>>> unpicking locally? >>>>>>> >>>>>>> Sent from my iPhone >>>>>>> >>>>>>>> On 12 May 2017, at 17:58, Clebert Suconic <[hidden email] >>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=1>> wrote: >>>>>>>> >>>>>>>> As the only thing that could affect this is the Change on timed >> buffer. >>>>>>>> Afaik >>>>>>>> >>>>>>>> >>>>>>>> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <[hidden email] >>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=2>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I'm considering only keeping the pooled buffer part and switch >> back to >>>>>> the >>>>>>>>> ole sleep or an improved sleep we had. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, May 12, 2017 at 12:49 PM Michael André Pearce < >>>>>>>>> [hidden email] <http:///user/SendEmail.jtp? >> type=node&node=4726091&i=3>> >>>>>> wrote: >>>>>>>>> >>>>>>>>>> As it seems I can't send images to mail list, just sent to you >> both >>>>>> via >>>>>>>>>> email. Some graphs we have comparing versions. >>>>>>>>>> >>>>>>>>>> Not sure what changes might cause it. >>>>>>>>>> >>>>>>>>>>> On 12 May 2017, at 17:37, Clebert Suconic <[hidden email] >>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=4>> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> There is a class we use on producer. TokenLimiter. Perhaps you >>>>>> could >>>>>>>>>>> reuse that one ? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz <[hidden email] >>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=5>> >>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> I was thinking of a similar solution but I've discovered that >>>>>> couldn't >>>>>>>>>> work >>>>>>>>>>>> (in the old or the new TimedBuffer too), because of the >>>>>>>>>>>> TimedBuffer::checkSize method that could force a flush if the >> batch >>>>>>>>>> buffer >>>>>>>>>>>> if not big enough to receive new data, going IOPS. >>>>>>>>>>>> Sadly TimedBuffer::checkSize is outside any timeout, but >> depends on >>>>>> the >>>>>>>>>>>> writers. >>>>>>>>>>>> >>>>>>>>>>>> That's why I've implemented the "compensation" right after any >>>>>> flush, >>>>>>>>>> in >>>>>>>>>>>> order to work with forced flushes too: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> https://github.com/franz1981/activemq-artemis/blob/ >>>>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/ >>>>>> src/main/java/org/apache/activemq/artemis/core/io/ >>>>>> buffer/TimedBuffer.java#L338 >>>>>>>>>>>> >>>>>>>>>>>> Doing it on TimedBuffer::flush all the flushes on disk will be >>>>>>>>>> compensated >>>>>>>>>>>> (half of the story: ASYNCIO is async so depends on libAIO >>>>>> partially!) >>>>>>>>>>>> Regarding the IOPS computation I've built this, as you've >>>>>> suggested: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> https://github.com/franz1981/activemq-artemis/blob/ >>>>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/ >>>>>> src/main/java/org/apache/activemq/artemis/core/io/ >>>>>> buffer/TimedBuffer.java#L119 >>>>>>>>>>>> >>>>>>>>>>>> The performance seems pretty good, it compensates well but it >> is >>>>>> faster >>>>>>>>>>>> than >>>>>>>>>>>> the original version, limiting IOPS too! >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Franz >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> View this message in context: >>>>>>>>>>>> >>>>>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- >>>>>> IOPS-Limiter-strategy-tp4725875p4726057.html >>>>>>>>>>>> Sent from the ActiveMQ - Dev mailing list archive at >> Nabble.com. >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Clebert Suconic >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> Clebert Suconic >>>>>>>>> >>>>>>>> -- >>>>>>>> Clebert Suconic >>>>>> >>>>>> >>>>>> ------------------------------ >>>>>> If you reply to this email, your message will be added to the >> discussion >>>>>> below: >>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- >>>>>> IOPS-Limiter-strategy-tp4725875p4726091.html >>>>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click >> here >>>>>> < >> >>>>>> . >>>>>> NAML >>>>>> <http://activemq.2283324.n4.nabble.com/template/ >> NamlServlet.jtp?macro=macro_viewer&id=instant_html% >> 21nabble%3Aemail.naml&base=nabble.naml.namespaces. >> BasicNamespace-nabble.view.web.template.NabbleNamespace- >> nabble.view.web.template.NodeNamespace&breadcrumbs= >> notify_subscribers%21nabble%3Aemail.naml-instant_emails% >> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >>>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> View this message in context: http://activemq.2283324.n4. >> nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726093.html >>>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com. >>> >>> >>> >>> -- >>> Clebert Suconic >> >> >> ------------------------------ >> If you reply to this email, your message will be added to the discussion >> below: >> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- >> IOPS-Limiter-strategy-tp4725875p4726097.html >> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click here >> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4725875&code=bmlncm8uZnJhQGdtYWlsLmNvbXw0NzI1ODc1fC0zNzI3NjE2NQ==> >> . >> NAML >> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> > > > > > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726099.html > Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
