Agree and this is something I can do pretty easily: currently I can maintain the original behaviour too just with a simple change in that class. But just out of curiosity Michael,the latencies are good then? I don't know what you get as throughput :P Because same/similar latencies but better throughout would be a dream :)
Il 13 mag 2017 6:11 AM, "Michael André Pearce" <[email protected]> ha scritto: > My request would be can we make it configurable via broker.xml to have new > and old behaviour. > > So that > > a) during testing it's easy to switch between the two solutions, instead > of having to deploy different versions :) > > b) existing deployments don't get any unexpected change in behaviour (or > at least can control the change, or revert it if any undesired effects > seen) > > > > > Sent from my iPhone > > > On 13 May 2017, at 01:58, Clebert Suconic <[hidden email] > <http:///user/SendEmail.jtp?type=node&node=4726106&i=0>> wrote: > > > > With Asyncio you have to flood requests a bit beyond the syncs per > second. > > That's why we calculate for aio separately. > > > > We may need to eventually add more threads to poll. > > > > But the good thing on aio is using less CPU. > > > > > > > >> On Fri, May 12, 2017 at 5:25 PM Francesco Nigro <[hidden email] > <http:///user/SendEmail.jtp?type=node&node=4726106&i=1>> wrote: > >> > >> And the most weird thing is that I'm having better results with NIO > then > >> ASYNCIO (with small sized messages) and I'm not getting why.... > >> > >> Time to go, 'm off; in Italy is 23:30 now..have fun guys and thanks: > always > >> great stuff :) > >> > >> 2017-05-12 23:02 GMT+02:00 nigro_franz <[hidden email] > <http:///user/SendEmail.jtp?type=node&node=4726106&i=2>>: > >> > >>> ahah np Michael!!! > >>> > >>> Anyway I was really expecting some more spike compared to the old > >>> TimedBuffer: Is one of the reasons why I wanted to put the IO > >> compensation. > >>> The new one seems to work well (with ASYNCIO) in this regard by the > tests > >>> I've done: I'm hoping it will help to remove that little spikes too! > >>> > >>> > >>> 2017-05-12 22:54 GMT+02:00 Michael André Pearce [via ActiveMQ] < > >>> [hidden email] <http:///user/SendEmail.jtp?type=node&node=4726106&i=3>>: > > >>> > >>>> Correct, :) seems like tester error. Apologies > >>>> > >>>> Sent from my iPhone > >>>> > >>>>> On 12 May 2017, at 21:24, nigro_franz <[hidden email] > >>>> <http:///user/SendEmail.jtp?type=node&node=4726101&i=0>> wrote: > >>>>> > >>>>> Hence the new TimedBuffer didn't change drastically the latencies? > >>>>> I've got some graph after the stress test with a maxed IO, if you're > >>>>> curious :) > >>>>> > >>>>> 2017-05-12 22:19 GMT+02:00 Michael André Pearce [via ActiveMQ] < > >>>>> [hidden email] <http:///user/SendEmail.jtp? > >>> type=node&node=4726101&i=1>>: > >>>> > >>>>> > >>>>>> Yup deleting the data on version change removes the issue. > >>>>>> > >>>>>> Sent from my iPhone > >>>>>> > >>>>>>> On 12 May 2017, at 20:54, Clebert Suconic <[hidden email] > >>>>>> <http:///user/SendEmail.jtp?type=node&node=4726097&i=0>> wrote: > >>>>>>> > >>>>>>> Can you cleanup your data (as a test only?) rm -rf data. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> perhaps the spikes you have is the compacting running? > >>>>>>> > >>>>>>> On Fri, May 12, 2017 at 3:51 PM, Michael André Pearce > >>>>>>> <[hidden email] <http:///user/SendEmail.jtp? > >>> type=node&node=4726097&i=1>> > >>>> > >>>>>> wrote: > >>>>>>>> All I'm doing is changing Artemis.profile of the instance to the > >>>>>> different versions. > >>>>>>>> > >>>>>>>> > >>>>>>>> Sent from my iPhone > >>>>>>>> > >>>>>>>>> On 12 May 2017, at 20:35, nigro_franz <[hidden email] > >>>>>> <http:///user/SendEmail.jtp?type=node&node=4726097&i=2>> wrote: > >>>>>>>>> > >>>>>>>>> Too strange...have you tried with NIO too?It uses always the > same > >>>>>> buffer to > >>>>>>>>> batch writes AFAIK.... > >>>>>>>>> > >>>>>>>>> 2017-05-12 21:30 GMT+02:00 Michael André Pearce [via ActiveMQ] < > >>>>>>>>> [hidden email] <http:///user/SendEmail.jtp? > >>>> type=node&node=4726097&i=3>>: > >>>>>> > >>>>>>>>> > >>>>>>>>>> Ok so it isn't the times buffer changes. Just deployed 2.1.0 > >> with > >>>> and > >>>>>>>>>> without the timed buffer changes. Running the same tests no > >>>>>> difference. > >>>>>>>>>> > >>>>>>>>>> Must be something else. Causing the behaviour change. > >>>>>>>>>> > >>>>>>>>>> Sent from my iPhone > >>>>>>>>>> > >>>>>>>>>>> On 12 May 2017, at 18:10, Michael André Pearce <[hidden email] > >>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=0>> > wrote: > >>>>>>>>>>> > >>>>>>>>>>> I've left for the day now. If I get time over the weekend I'll > >>> try > >>>>>> see > >>>>>>>>>> if I can make a build of 2.1.0 without that change and see if > it > >>>>>> makes any > >>>>>>>>>> difference. > >>>>>>>>>>> > >>>>>>>>>>> Can someone point me to the PR for that change, so I know what > >>> I'm > >>>>>>>>>> unpicking locally? > >>>>>>>>>>> > >>>>>>>>>>> Sent from my iPhone > >>>>>>>>>>> > >>>>>>>>>>>> On 12 May 2017, at 17:58, Clebert Suconic <[hidden email] > >>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=1>> > wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> As the only thing that could affect this is the Change on > >> timed > >>>>>> buffer. > >>>>>>>>>>>> Afaik > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <[hidden > >> email] > >>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=2>> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> I'm considering only keeping the pooled buffer part and > >> switch > >>>>>> back to > >>>>>>>>>> the > >>>>>>>>>>>>> ole sleep or an improved sleep we had. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Fri, May 12, 2017 at 12:49 PM Michael André Pearce < > >>>>>>>>>>>>> [hidden email] <http:///user/SendEmail.jtp? > >>>>>> type=node&node=4726091&i=3>> > >>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> As it seems I can't send images to mail list, just sent to > >>>> you > >>>>>> both > >>>>>>>>>> via > >>>>>>>>>>>>>> email. Some graphs we have comparing versions. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Not sure what changes might cause it. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On 12 May 2017, at 17:37, Clebert Suconic <[hidden email] > >>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=4>> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> There is a class we use on producer. TokenLimiter. > >> Perhaps > >>>> you > >>>>>>>>>> could > >>>>>>>>>>>>>>> reuse that one ? > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz <[hidden > >> email] > >>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=5>> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> I was thinking of a similar solution but I've discovered > >>> that > >>>>>>>>>> couldn't > >>>>>>>>>>>>>> work > >>>>>>>>>>>>>>>> (in the old or the new TimedBuffer too), because of the > >>>>>>>>>>>>>>>> TimedBuffer::checkSize method that could force a flush if > >>> the > >>>>>> batch > >>>>>>>>>>>>>> buffer > >>>>>>>>>>>>>>>> if not big enough to receive new data, going IOPS. > >>>>>>>>>>>>>>>> Sadly TimedBuffer::checkSize is outside any timeout, but > >>>>>> depends on > >>>>>>>>>> the > >>>>>>>>>>>>>>>> writers. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> That's why I've implemented the "compensation" right > after > >>>> any > >>>>>>>>>> flush, > >>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>> order to work with forced flushes too: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>> https://github.com/franz1981/activemq-artemis/blob/ > >>>>>>>>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/ > >>>>>>>>>> src/main/java/org/apache/activemq/artemis/core/io/ > >>>>>>>>>> buffer/TimedBuffer.java#L338 > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Doing it on TimedBuffer::flush all the flushes on disk > >> will > >>>> be > >>>>>>>>>>>>>> compensated > >>>>>>>>>>>>>>>> (half of the story: ASYNCIO is async so depends on libAIO > >>>>>>>>>> partially!) > >>>>>>>>>>>>>>>> Regarding the IOPS computation I've built this, as you've > >>>>>>>>>> suggested: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>> https://github.com/franz1981/activemq-artemis/blob/ > >>>>>>>>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/ > >>>>>>>>>> src/main/java/org/apache/activemq/artemis/core/io/ > >>>>>>>>>> buffer/TimedBuffer.java#L119 > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> The performance seems pretty good, it compensates well > but > >>> it > >>>>>> is > >>>>>>>>>> faster > >>>>>>>>>>>>>>>> than > >>>>>>>>>>>>>>>> the original version, limiting IOPS too! > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> Franz > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>> View this message in context: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- > >>>>>>>>>> IOPS-Limiter-strategy-tp4725875p4726057.html > >>>>>>>>>>>>>>>> Sent from the ActiveMQ - Dev mailing list archive at > >>>>>> Nabble.com. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>> Clebert Suconic > >>>>>>>>>>>>>> > >>>>>>>>>>>>> -- > >>>>>>>>>>>>> Clebert Suconic > >>>>>>>>>>>>> > >>>>>>>>>>>> -- > >>>>>>>>>>>> Clebert Suconic > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> ------------------------------ > >>>>>>>>>> If you reply to this email, your message will be added to the > >>>>>> discussion > >>>>>>>>>> below: > >>>>>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- > >>>>>>>>>> IOPS-Limiter-strategy-tp4725875p4726091.html > >>>>>>>>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, > >> click > >>>>>> here > >>>>>>>>>> < > >>>>>> > >>>>>>>>>> . > >>>>>>>>>> NAML > >>>>>>>>>> <http://activemq.2283324.n4.nabble.com/template/ > >>>>>> NamlServlet.jtp?macro=macro_viewer&id=instant_html% > >>>>>> 21nabble%3Aemail.naml&base=nabble.naml.namespaces. > >>>>>> BasicNamespace-nabble.view.web.template.NabbleNamespace- > >>>>>> nabble.view.web.template.NodeNamespace&breadcrumbs= > >>>>>> notify_subscribers%21nabble%3Aemail.naml-instant_emails% > >>>>>> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> View this message in context: http://activemq.2283324.n4. > >>>>>> nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy- > >>> tp4725875p4726093.html > >>>>>>>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Clebert Suconic > >>>>>> > >>>>>> > >>>>>> ------------------------------ > >>>>>> If you reply to this email, your message will be added to the > >>>> discussion > >>>>>> below: > >>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- > >>>>>> IOPS-Limiter-strategy-tp4725875p4726097.html > >>>>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click > >>> here > >>>>>> < > >>>>>> . > >>>>>> NAML > >>>>>> <http://activemq.2283324.n4.nabble.com/template/ > >>>> NamlServlet.jtp?macro=macro_viewer&id=instant_html% > >>>> 21nabble%3Aemail.naml&base=nabble.naml.namespaces. > >>>> BasicNamespace-nabble.view.web.template.NabbleNamespace- > >>>> nabble.view.web.template.NodeNamespace&breadcrumbs= > >>>> notify_subscribers%21nabble%3Aemail.naml-instant_emails% > >>>> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> View this message in context: http://activemq.2283324.n4. > >>>> > >> nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726099.html > >>>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com. > >>>> > >>>> > >>>> ------------------------------ > >>>> If you reply to this email, your message will be added to the > >> discussion > >>>> below: > >>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- > >>>> IOPS-Limiter-strategy-tp4725875p4726101.html > >>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click > here > >>>> <http://activemq.2283324.n4.nabble.com/template/ > NamlServlet.jtp?macro= > >>> unsubscribe_by_code&node=4725875&code=bmlncm8uZnJhQGdtYWlsLmNvbXw0Nz > >>> I1ODc1fC0zNzI3NjE2NQ==> > >>>> . > >>>> NAML > >>>> <http://activemq.2283324.n4.nabble.com/template/ > >>> NamlServlet.jtp?macro=macro_viewer&id=instant_html% > >>> 21nabble%3Aemail.naml&base=nabble.naml.namespaces. > >>> BasicNamespace-nabble.view.web.template.NabbleNamespace- > >>> nabble.view.web.template.NodeNamespace&breadcrumbs= > >>> notify_subscribers%21nabble%3Aemail.naml-instant_emails% > >>> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > >>>> > >>> > >>> > >>> > >>> > >>> -- > >>> View this message in context: http://activemq.2283324.n4. > >>> nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy- > tp4725875p4726102.html > >>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com. > >>> > >> > > -- > > Clebert Suconic > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis- > IOPS-Limiter-strategy-tp4725875p4726106.html > To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click here > <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4725875&code=bmlncm8uZnJhQGdtYWlsLmNvbXw0NzI1ODc1fC0zNzI3NjE2NQ==> > . > NAML > <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >
