Hi Asanka, For AMQP yes we need to iterate. Since we need to block each channel separately. At the moment with TCP back pressure for MQTT, what Pamod is suggesting is to block all the publishers at once. With a separate listener for global flow control we can give the protocol developer the ability do his own implementation when Andes core notifies global flow control limit is reached.
@Pamod correct me if I misunderstood the use-case. Thanks, Asitha On Fri, Jun 5, 2015 at 3:24 PM, Asanka Abeyweera <[email protected]> wrote: > Hi Asitha, > > Event with this approach, at the end doesn't it boils down to iterating > over a list to inform the listeners. > > > > On Fri, Jun 5, 2015 at 3:03 PM, Asitha Nanayakkara <[email protected]> > wrote: > >> Hi Pamod, >> >> In Andes core, we can have a listener to inform global flow control. >> Similar to FlowControlListner. With that when the global flow control limit >> is reached we can invoke this listener interface. In the protocol level >> implementation of this global flow control listener interface, we can >> either iterate or directly flow control all publishers depending on the >> protocol capability. WDYT? >> >> Thanks, >> Asitha >> >> On Fri, Jun 5, 2015 at 2:45 PM, Pamod Sylvester <[email protected]> wrote: >> >>> Hi All, >>> >>> Flow control is activated when the distruptor buffer is full, database >>> encounters an error etc. When its being applied, currently it iterates >>> through the list of publisher channels, notifying the publishers >>> individually to flow control. >>> >>> in the FlowControlManager its illustrated through the following code >>> segment. >>> >>> private synchronized void blockListenersOnBufferBasedFlowControl() { >>> if (!globalBufferBasedFlowControlEnabled) { >>> globalBufferBasedFlowControlEnabled = true; >>> >>> for (AndesChannel channel : channels) { >>> channel.notifyGlobalBufferBasedFlowControlActivation(); >>> } >>> >>> scheduledBufferBasedFlowControlTimeoutFuture = >>> executor.schedule(flowControlTimeoutTask, 1, TimeUnit.MINUTES); >>> log.info("Global buffer based flow control enabled."); >>> } >>> } >>> >>> >>> Since its being iterated and flow controlled individually there could be >>> a slight latency when applying flow control to the last publisher in the >>> list. (Until flow control is applied the publisher/s would continue to send >>> messages) >>> >>> The question is, lets say there're > 10000 of publishers which need to >>> be flow controlled at a given time. Will there be a possibility for the >>> broker to go OOM due to the above mentioned reason. i.e flow control is >>> applied due to buffer nearly became full ? >>> >>> If that's the case, what would be the best way to deal with it ? >>> >>> One suggestion would be to at a glance, stop the server from accepting >>> messages totally.( i.e in Netty server we could do it by adding a child >>> option to set auto read 'false' in the ServerBootstrap). until the whole >>> publisher list is iterated and flow control is applied and then retain >>> accepting connections. >>> >>> Thanks, >>> Pamod >>> >>> -- >>> *Pamod Sylvester * >>> >>> *WSO2 Inc.; http://wso2.com <http://wso2.com>* >>> cell: +94 77 7779495 >>> >> >> >> >> -- >> *Asitha Nanayakkara* >> Software Engineer >> WSO2, Inc. http://wso2.com/ >> Mob: + 94 77 85 30 682 >> >> > > > -- > Asanka Abeyweera > Software Engineer > WSO2 Inc. > > Phone: +94 712228648 > Blog: a5anka.github.io > -- *Asitha Nanayakkara* Software Engineer WSO2, Inc. http://wso2.com/ Mob: + 94 77 85 30 682
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
