With Asyncio you have to flood requests a bit beyond the syncs per second. That's why we calculate for aio separately.
We may need to eventually add more threads to poll. But the good thing on aio is using less CPU. On Fri, May 12, 2017 at 5:25 PM Francesco Nigro <[email protected]> wrote: > And the most weird thing is that I'm having better results with NIO then > ASYNCIO (with small sized messages) and I'm not getting why.... > > Time to go, 'm off; in Italy is 23:30 now..have fun guys and thanks: always > great stuff :) > > 2017-05-12 23:02 GMT+02:00 nigro_franz <[email protected]>: > > > ahah np Michael!!! > > > > Anyway I was really expecting some more spike compared to the old > > TimedBuffer: Is one of the reasons why I wanted to put the IO > compensation. > > The new one seems to work well (with ASYNCIO) in this regard by the tests > > I've done: I'm hoping it will help to remove that little spikes too! > > > > > > 2017-05-12 22:54 GMT+02:00 Michael André Pearce [via ActiveMQ] < > > [email protected]>: > > > > > Correct, :) seems like tester error. Apologies > > > > > > Sent from my iPhone > > > > > > > On 12 May 2017, at 21:24, nigro_franz <[hidden email] > > > <http:///user/SendEmail.jtp?type=node&node=4726101&i=0>> wrote: > > > > > > > > Hence the new TimedBuffer didn't change drastically the latencies? > > > > I've got some graph after the stress test with a maxed IO, if you're > > > > curious :) > > > > > > > > 2017-05-12 22:19 GMT+02:00 Michael André Pearce [via ActiveMQ] < > > > > [hidden email] <http:///user/SendEmail.jtp? > > type=node&node=4726101&i=1>>: > > > > > > > > > > >> Yup deleting the data on version change removes the issue. > > > >> > > > >> Sent from my iPhone > > > >> > > > >>> On 12 May 2017, at 20:54, Clebert Suconic <[hidden email] > > > >> <http:///user/SendEmail.jtp?type=node&node=4726097&i=0>> wrote: > > > >>> > > > >>> Can you cleanup your data (as a test only?) rm -rf data. > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> perhaps the spikes you have is the compacting running? > > > >>> > > > >>> On Fri, May 12, 2017 at 3:51 PM, Michael André Pearce > > > >>> <[hidden email] <http:///user/SendEmail.jtp? > > type=node&node=4726097&i=1>> > > > > > > >> wrote: > > > >>>> All I'm doing is changing Artemis.profile of the instance to the > > > >> different versions. > > > >>>> > > > >>>> > > > >>>> Sent from my iPhone > > > >>>> > > > >>>>> On 12 May 2017, at 20:35, nigro_franz <[hidden email] > > > >> <http:///user/SendEmail.jtp?type=node&node=4726097&i=2>> wrote: > > > >>>>> > > > >>>>> Too strange...have you tried with NIO too?It uses always the same > > > >> buffer to > > > >>>>> batch writes AFAIK.... > > > >>>>> > > > >>>>> 2017-05-12 21:30 GMT+02:00 Michael André Pearce [via ActiveMQ] < > > > >>>>> [hidden email] <http:///user/SendEmail.jtp? > > > type=node&node=4726097&i=3>>: > > > >> > > > >>>>> > > > >>>>>> Ok so it isn't the times buffer changes. Just deployed 2.1.0 > with > > > and > > > >>>>>> without the timed buffer changes. Running the same tests no > > > >> difference. > > > >>>>>> > > > >>>>>> Must be something else. Causing the behaviour change. > > > >>>>>> > > > >>>>>> Sent from my iPhone > > > >>>>>> > > > >>>>>>> On 12 May 2017, at 18:10, Michael André Pearce <[hidden email] > > > >>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=0>> wrote: > > > >>>>>>> > > > >>>>>>> I've left for the day now. If I get time over the weekend I'll > > try > > > >> see > > > >>>>>> if I can make a build of 2.1.0 without that change and see if it > > > >> makes any > > > >>>>>> difference. > > > >>>>>>> > > > >>>>>>> Can someone point me to the PR for that change, so I know what > > I'm > > > >>>>>> unpicking locally? > > > >>>>>>> > > > >>>>>>> Sent from my iPhone > > > >>>>>>> > > > >>>>>>>> On 12 May 2017, at 17:58, Clebert Suconic <[hidden email] > > > >>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=1>> wrote: > > > >>>>>>>> > > > >>>>>>>> As the only thing that could affect this is the Change on > timed > > > >> buffer. > > > >>>>>>>> Afaik > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <[hidden > email] > > > >>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=2>> > > > >>>>>>>> wrote: > > > >>>>>>>> > > > >>>>>>>>> I'm considering only keeping the pooled buffer part and > switch > > > >> back to > > > >>>>>> the > > > >>>>>>>>> ole sleep or an improved sleep we had. > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> On Fri, May 12, 2017 at 12:49 PM Michael André Pearce < > > > >>>>>>>>> [hidden email] <http:///user/SendEmail.jtp? > > > >> type=node&node=4726091&i=3>> > > > >>>>>> wrote: > > > >>>>>>>>> > > > >>>>>>>>>> As it seems I can't send images to mail list, just sent to > > > you > > > >> both > > > >>>>>> via > > > >>>>>>>>>> email. Some graphs we have comparing versions. > > > >>>>>>>>>> > > > >>>>>>>>>> Not sure what changes might cause it. > > > >>>>>>>>>> > > > >>>>>>>>>>> On 12 May 2017, at 17:37, Clebert Suconic <[hidden email] > > > >>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=4>> > > > >>>>>>>>>> wrote: > > > >>>>>>>>>>> > > > >>>>>>>>>>> There is a class we use on producer. TokenLimiter. > Perhaps > > > you > > > >>>>>> could > > > >>>>>>>>>>> reuse that one ? > > > >>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz <[hidden > email] > > > >>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=5>> > > > >>>>>>>>>> wrote: > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> I was thinking of a similar solution but I've discovered > > that > > > >>>>>> couldn't > > > >>>>>>>>>> work > > > >>>>>>>>>>>> (in the old or the new TimedBuffer too), because of the > > > >>>>>>>>>>>> TimedBuffer::checkSize method that could force a flush if > > the > > > >> batch > > > >>>>>>>>>> buffer > > > >>>>>>>>>>>> if not big enough to receive new data, going IOPS. > > > >>>>>>>>>>>> Sadly TimedBuffer::checkSize is outside any timeout, but > > > >> depends on > > > >>>>>> the > > > >>>>>>>>>>>> writers. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> That's why I've implemented the "compensation" right after > > > any > > > >>>>>> flush, > > > >>>>>>>>>> in > > > >>>>>>>>>>>> order to work with forced flushes too: > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>> https://github.com/franz1981/activemq-artemis/blob/ > > > >>>>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/ > > > >>>>>> src/main/java/org/apache/activemq/artemis/core/io/ > > > >>>>>> buffer/TimedBuffer.java#L338 > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Doing it on TimedBuffer::flush all the flushes on disk > will > > > be > > > >>>>>>>>>> compensated > > > >>>>>>>>>>>> (half of the story: ASYNCIO is async so depends on libAIO > > > >>>>>> partially!) > > > >>>>>>>>>>>> Regarding the IOPS computation I've built this, as you've > > > >>>>>> suggested: > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>> https://github.com/franz1981/activemq-artemis/blob/ > > > >>>>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/ > > > >>>>>> src/main/java/org/apache/activemq/artemis/core/io/ > > > >>>>>> buffer/TimedBuffer.java#L119 > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> The performance seems pretty good, it compensates well but > > it > > > >> is > > > >>>>>> faster > > > >>>>>>>>>>>> than > > > >>>>>>>>>>>> the original version, limiting IOPS too! > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Thanks, > > > >>>>>>>>>>>> Franz > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> -- > > > >>>>>>>>>>>> View this message in context: > > > >>>>>>>>>>>> > > > >>>>>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- > > > >>>>>> IOPS-Limiter-strategy-tp4725875p4726057.html > > > >>>>>>>>>>>> Sent from the ActiveMQ - Dev mailing list archive at > > > >> Nabble.com. > > > >>>>>>>>>>>> > > > >>>>>>>>>>> -- > > > >>>>>>>>>>> Clebert Suconic > > > >>>>>>>>>> > > > >>>>>>>>> -- > > > >>>>>>>>> Clebert Suconic > > > >>>>>>>>> > > > >>>>>>>> -- > > > >>>>>>>> Clebert Suconic > > > >>>>>> > > > >>>>>> > > > >>>>>> ------------------------------ > > > >>>>>> If you reply to this email, your message will be added to the > > > >> discussion > > > >>>>>> below: > > > >>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- > > > >>>>>> IOPS-Limiter-strategy-tp4725875p4726091.html > > > >>>>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, > click > > > >> here > > > >>>>>> < > > > >> > > > >>>>>> . > > > >>>>>> NAML > > > >>>>>> <http://activemq.2283324.n4.nabble.com/template/ > > > >> NamlServlet.jtp?macro=macro_viewer&id=instant_html% > > > >> 21nabble%3Aemail.naml&base=nabble.naml.namespaces. > > > >> BasicNamespace-nabble.view.web.template.NabbleNamespace- > > > >> nabble.view.web.template.NodeNamespace&breadcrumbs= > > > >> notify_subscribers%21nabble%3Aemail.naml-instant_emails% > > > >> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > >>>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> -- > > > >>>>> View this message in context: http://activemq.2283324.n4. > > > >> nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy- > > tp4725875p4726093.html > > > >>>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com. > > > >>> > > > >>> > > > >>> > > > >>> -- > > > >>> Clebert Suconic > > > >> > > > >> > > > >> ------------------------------ > > > >> If you reply to this email, your message will be added to the > > > discussion > > > >> below: > > > >> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- > > > >> IOPS-Limiter-strategy-tp4725875p4726097.html > > > >> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click > > here > > > >> < > > > >> . > > > >> NAML > > > >> <http://activemq.2283324.n4.nabble.com/template/ > > > NamlServlet.jtp?macro=macro_viewer&id=instant_html% > > > 21nabble%3Aemail.naml&base=nabble.naml.namespaces. > > > BasicNamespace-nabble.view.web.template.NabbleNamespace- > > > nabble.view.web.template.NodeNamespace&breadcrumbs= > > > notify_subscribers%21nabble%3Aemail.naml-instant_emails% > > > 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > >> > > > > > > > > > > > > > > > > > > > > -- > > > > View this message in context: http://activemq.2283324.n4. > > > > nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726099.html > > > > Sent from the ActiveMQ - Dev mailing list archive at Nabble.com. > > > > > > > > > ------------------------------ > > > If you reply to this email, your message will be added to the > discussion > > > below: > > > http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- > > > IOPS-Limiter-strategy-tp4725875p4726101.html > > > To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click here > > > <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro= > > unsubscribe_by_code&node=4725875&code=bmlncm8uZnJhQGdtYWlsLmNvbXw0Nz > > I1ODc1fC0zNzI3NjE2NQ==> > > > . > > > NAML > > > <http://activemq.2283324.n4.nabble.com/template/ > > NamlServlet.jtp?macro=macro_viewer&id=instant_html% > > 21nabble%3Aemail.naml&base=nabble.naml.namespaces. > > BasicNamespace-nabble.view.web.template.NabbleNamespace- > > nabble.view.web.template.NodeNamespace&breadcrumbs= > > notify_subscribers%21nabble%3Aemail.naml-instant_emails% > > 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > > > > > > > > > > > -- > > View this message in context: http://activemq.2283324.n4. > > nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726102.html > > Sent from the ActiveMQ - Dev mailing list archive at Nabble.com. > > > -- Clebert Suconic
