These are the throughput too (avg), but meaningful to understand how are the changes:
OLD_TIMED_BUFFER: 324 ops/sec NEW_TIMED_BUFFER: 471 ops/sec IO_LIMITED_NEW_TIMED_BUFFER: 525 ops/sec this is what I'm getting: [image: Immagine incorporata 1] While considering until the 99%: [image: Immagine incorporata 2] Throughput aside, I'm not seeing a huge difference between the 3 implementations until 99,99% For sure the old one was the more stable. but it has a great difference in the throughput too, so the same number of messages can be send in less time, making a burst much shorter. Running for the same time the 3 implementation is not fair I think, just my 2 cents... 2017-05-12 22:36 GMT+02:00 Francesco Nigro <[email protected]>: > Hence the new TimedBuffer didn't change drastically the latencies? > I've got some graph after the stress test with a maxed IO, if you're > curious :) > > 2017-05-12 22:19 GMT+02:00 Michael André Pearce [via ActiveMQ] < > [email protected]>: > >> Yup deleting the data on version change removes the issue. >> >> Sent from my iPhone >> >> > On 12 May 2017, at 20:54, Clebert Suconic <[hidden email] >> <http:///user/SendEmail.jtp?type=node&node=4726097&i=0>> wrote: >> > >> > Can you cleanup your data (as a test only?) rm -rf data. >> > >> > >> > >> > >> > perhaps the spikes you have is the compacting running? >> > >> > On Fri, May 12, 2017 at 3:51 PM, Michael André Pearce >> > <[hidden email] <http:///user/SendEmail.jtp?type=node&node=4726097&i=1>> >> wrote: >> >> All I'm doing is changing Artemis.profile of the instance to the >> different versions. >> >> >> >> >> >> Sent from my iPhone >> >> >> >>> On 12 May 2017, at 20:35, nigro_franz <[hidden email] >> <http:///user/SendEmail.jtp?type=node&node=4726097&i=2>> wrote: >> >>> >> >>> Too strange...have you tried with NIO too?It uses always the same >> buffer to >> >>> batch writes AFAIK.... >> >>> >> >>> 2017-05-12 21:30 GMT+02:00 Michael André Pearce [via ActiveMQ] < >> >>> [hidden email] >> <http:///user/SendEmail.jtp?type=node&node=4726097&i=3>>: >> >>> >> >>>> Ok so it isn't the times buffer changes. Just deployed 2.1.0 with >> and >> >>>> without the timed buffer changes. Running the same tests no >> difference. >> >>>> >> >>>> Must be something else. Causing the behaviour change. >> >>>> >> >>>> Sent from my iPhone >> >>>> >> >>>>> On 12 May 2017, at 18:10, Michael André Pearce <[hidden email] >> >>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=0>> wrote: >> >>>>> >> >>>>> I've left for the day now. If I get time over the weekend I'll try >> see >> >>>> if I can make a build of 2.1.0 without that change and see if it >> makes any >> >>>> difference. >> >>>>> >> >>>>> Can someone point me to the PR for that change, so I know what I'm >> >>>> unpicking locally? >> >>>>> >> >>>>> Sent from my iPhone >> >>>>> >> >>>>>> On 12 May 2017, at 17:58, Clebert Suconic <[hidden email] >> >>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=1>> wrote: >> >>>>>> >> >>>>>> As the only thing that could affect this is the Change on timed >> buffer. >> >>>>>> Afaik >> >>>>>> >> >>>>>> >> >>>>>> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <[hidden email] >> >>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=2>> >> >>>>>> wrote: >> >>>>>> >> >>>>>>> I'm considering only keeping the pooled buffer part and switch >> back to >> >>>> the >> >>>>>>> ole sleep or an improved sleep we had. >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> On Fri, May 12, 2017 at 12:49 PM Michael André Pearce < >> >>>>>>> [hidden email] <http:///user/SendEmail.jtp?ty >> pe=node&node=4726091&i=3>> >> >>>> wrote: >> >>>>>>> >> >>>>>>>> As it seems I can't send images to mail list, just sent to you >> both >> >>>> via >> >>>>>>>> email. Some graphs we have comparing versions. >> >>>>>>>> >> >>>>>>>> Not sure what changes might cause it. >> >>>>>>>> >> >>>>>>>>> On 12 May 2017, at 17:37, Clebert Suconic <[hidden email] >> >>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=4>> >> >>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>> There is a class we use on producer. TokenLimiter. Perhaps >> you >> >>>> could >> >>>>>>>>> reuse that one ? >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz <[hidden email] >> >>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=5>> >> >>>>>>>> wrote: >> >>>>>>>>>> >> >>>>>>>>>> I was thinking of a similar solution but I've discovered that >> >>>> couldn't >> >>>>>>>> work >> >>>>>>>>>> (in the old or the new TimedBuffer too), because of the >> >>>>>>>>>> TimedBuffer::checkSize method that could force a flush if the >> batch >> >>>>>>>> buffer >> >>>>>>>>>> if not big enough to receive new data, going IOPS. >> >>>>>>>>>> Sadly TimedBuffer::checkSize is outside any timeout, but >> depends on >> >>>> the >> >>>>>>>>>> writers. >> >>>>>>>>>> >> >>>>>>>>>> That's why I've implemented the "compensation" right after any >> >>>> flush, >> >>>>>>>> in >> >>>>>>>>>> order to work with forced flushes too: >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>> https://github.com/franz1981/activemq-artemis/blob/ >> >>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/ >> >>>> src/main/java/org/apache/activemq/artemis/core/io/ >> >>>> buffer/TimedBuffer.java#L338 >> >>>>>>>>>> >> >>>>>>>>>> Doing it on TimedBuffer::flush all the flushes on disk will be >> >>>>>>>> compensated >> >>>>>>>>>> (half of the story: ASYNCIO is async so depends on libAIO >> >>>> partially!) >> >>>>>>>>>> Regarding the IOPS computation I've built this, as you've >> >>>> suggested: >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>> https://github.com/franz1981/activemq-artemis/blob/ >> >>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/ >> >>>> src/main/java/org/apache/activemq/artemis/core/io/ >> >>>> buffer/TimedBuffer.java#L119 >> >>>>>>>>>> >> >>>>>>>>>> The performance seems pretty good, it compensates well but it >> is >> >>>> faster >> >>>>>>>>>> than >> >>>>>>>>>> the original version, limiting IOPS too! >> >>>>>>>>>> >> >>>>>>>>>> Thanks, >> >>>>>>>>>> Franz >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> -- >> >>>>>>>>>> View this message in context: >> >>>>>>>>>> >> >>>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- >> >>>> IOPS-Limiter-strategy-tp4725875p4726057.html >> >>>>>>>>>> Sent from the ActiveMQ - Dev mailing list archive at >> Nabble.com. >> >>>>>>>>>> >> >>>>>>>>> -- >> >>>>>>>>> Clebert Suconic >> >>>>>>>> >> >>>>>>> -- >> >>>>>>> Clebert Suconic >> >>>>>>> >> >>>>>> -- >> >>>>>> Clebert Suconic >> >>>> >> >>>> >> >>>> ------------------------------ >> >>>> If you reply to this email, your message will be added to the >> discussion >> >>>> below: >> >>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- >> >>>> IOPS-Limiter-strategy-tp4725875p4726091.html >> >>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click >> here >> >>>> < >> >> >>>> . >> >>>> NAML >> >>>> <http://activemq.2283324.n4.nabble.com/template/NamlServlet. >> jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail. >> naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template. >> NabbleNamespace-nabble.view.web.template.NodeNamespace& >> breadcrumbs=notify_subscribers%21nabble%3Aemail. >> naml-instant_emails%21nabble%3Aemail.naml-send_instant_ >> email%21nabble%3Aemail.naml> >> >>>> >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> View this message in context: http://activemq.2283324.n4.nab >> ble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726093.html >> >>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com. >> > >> > >> > >> > -- >> > Clebert Suconic >> >> >> ------------------------------ >> If you reply to this email, your message will be added to the discussion >> below: >> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS- >> Limiter-strategy-tp4725875p4726097.html >> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click here >> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4725875&code=bmlncm8uZnJhQGdtYWlsLmNvbXw0NzI1ODc1fC0zNzI3NjE2NQ==> >> . >> NAML >> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> > > image.png (74K) <http://activemq.2283324.n4.nabble.com/attachment/4726100/0/image.png> image.png (87K) <http://activemq.2283324.n4.nabble.com/attachment/4726100/1/image.png> -- View this message in context: http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726100.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
