Yes. when we stop auto read we won't be able to read. But in MQTT there is no flow control mechanism. Hence, we can use stop auto read to do flow control (publishers) And I agree this may not work in AMQP since we have the flow control mechanism within the protocol itself. We may need to send some packets back and forth.
What I suggest is to give that extra bit of information (global flow control is enabled or disabled) to the protocol so that at protocol level we can take advantage of that information if that is applicable. If not we may have to iterate through all the publishers. On Fri, Jun 5, 2015 at 4:51 PM, Asanka Abeyweera <[email protected]> wrote: > Hi Asitha, > > Thank you for the explanation. Say we go ahead an stop auto read in Netty > (or Mina) when global flow control triggered. Will it be an issue to send > the flow control enable messages to publishers since some packets have to > go back and forth? > > On Fri, Jun 5, 2015 at 4:35 PM, Asitha Nanayakkara <[email protected]> > wrote: > >> Hi Asanka, >> >> For AMQP yes we need to iterate. Since we need to block each channel >> separately. At the moment with TCP back pressure for MQTT, what Pamod is >> suggesting is to block all the publishers at once. With a separate listener >> for global flow control we can give the protocol developer the ability do >> his own implementation when Andes core notifies global flow control limit >> is reached. >> >> @Pamod correct me if I misunderstood the use-case. >> >> Thanks, >> Asitha >> >> >> On Fri, Jun 5, 2015 at 3:24 PM, Asanka Abeyweera <[email protected]> >> wrote: >> >>> Hi Asitha, >>> >>> Event with this approach, at the end doesn't it boils down to iterating >>> over a list to inform the listeners. >>> >>> >>> >>> On Fri, Jun 5, 2015 at 3:03 PM, Asitha Nanayakkara <[email protected]> >>> wrote: >>> >>>> Hi Pamod, >>>> >>>> In Andes core, we can have a listener to inform global flow control. >>>> Similar to FlowControlListner. With that when the global flow control limit >>>> is reached we can invoke this listener interface. In the protocol level >>>> implementation of this global flow control listener interface, we can >>>> either iterate or directly flow control all publishers depending on the >>>> protocol capability. WDYT? >>>> >>>> Thanks, >>>> Asitha >>>> >>>> On Fri, Jun 5, 2015 at 2:45 PM, Pamod Sylvester <[email protected]> wrote: >>>> >>>>> Hi All, >>>>> >>>>> Flow control is activated when the distruptor buffer is full, database >>>>> encounters an error etc. When its being applied, currently it iterates >>>>> through the list of publisher channels, notifying the publishers >>>>> individually to flow control. >>>>> >>>>> in the FlowControlManager its illustrated through the following code >>>>> segment. >>>>> >>>>> private synchronized void blockListenersOnBufferBasedFlowControl() >>>>> { >>>>> if (!globalBufferBasedFlowControlEnabled) { >>>>> globalBufferBasedFlowControlEnabled = true; >>>>> >>>>> for (AndesChannel channel : channels) { >>>>> channel.notifyGlobalBufferBasedFlowControlActivation(); >>>>> } >>>>> >>>>> scheduledBufferBasedFlowControlTimeoutFuture = >>>>> executor.schedule(flowControlTimeoutTask, 1, TimeUnit.MINUTES); >>>>> log.info("Global buffer based flow control enabled."); >>>>> } >>>>> } >>>>> >>>>> >>>>> Since its being iterated and flow controlled individually there could >>>>> be a slight latency when applying flow control to the last publisher in >>>>> the >>>>> list. (Until flow control is applied the publisher/s would continue to >>>>> send >>>>> messages) >>>>> >>>>> The question is, lets say there're > 10000 of publishers which need to >>>>> be flow controlled at a given time. Will there be a possibility for the >>>>> broker to go OOM due to the above mentioned reason. i.e flow control is >>>>> applied due to buffer nearly became full ? >>>>> >>>>> If that's the case, what would be the best way to deal with it ? >>>>> >>>>> One suggestion would be to at a glance, stop the server from accepting >>>>> messages totally.( i.e in Netty server we could do it by adding a child >>>>> option to set auto read 'false' in the ServerBootstrap). until the whole >>>>> publisher list is iterated and flow control is applied and then retain >>>>> accepting connections. >>>>> >>>>> Thanks, >>>>> Pamod >>>>> >>>>> -- >>>>> *Pamod Sylvester * >>>>> >>>>> *WSO2 Inc.; http://wso2.com <http://wso2.com>* >>>>> cell: +94 77 7779495 >>>>> >>>> >>>> >>>> >>>> -- >>>> *Asitha Nanayakkara* >>>> Software Engineer >>>> WSO2, Inc. http://wso2.com/ >>>> Mob: + 94 77 85 30 682 >>>> >>>> >>> >>> >>> -- >>> Asanka Abeyweera >>> Software Engineer >>> WSO2 Inc. >>> >>> Phone: +94 712228648 >>> Blog: a5anka.github.io >>> >> >> >> >> -- >> *Asitha Nanayakkara* >> Software Engineer >> WSO2, Inc. http://wso2.com/ >> Mob: + 94 77 85 30 682 >> >> > > > -- > Asanka Abeyweera > Software Engineer > WSO2 Inc. > > Phone: +94 712228648 > Blog: a5anka.github.io > -- *Asitha Nanayakkara* Software Engineer WSO2, Inc. http://wso2.com/ Mob: + 94 77 85 30 682
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
