This needs to be implemented for topic matching as well. On Tue, Dec 16, 2014 at 9:58 AM, Pamod Sylvester <[email protected]> wrote: > > Sure, let's discuss and see how we could incorporate this algorithm, This > could also be added into the Andes Kernal as well. > > Thanks, > Pamod > > On Tue, Dec 16, 2014 at 7:54 AM, Asitha Nanayakkara <[email protected]> > wrote: >> >> Hi Pamod, >> >> Inverted bitmap technique[1] looks good for the purpose. Algorithm is >> O(n) where n is number of sections in the destination, not subscribers. >> >> Only concern is memory consumption and updating the maps when new >> subscribers come while messages are flowing through the system at higher >> rates (This will introduce some pauses to delivery). Since adding and >> removing subscriptions doesn't happen frequently we can live with that I >> think. Need to look at the memory consumption to make sure this is >> feasible. >> >> [1] http://zeromq.org/whitepapers:message-matching >> >> On Mon, Dec 15, 2014 at 5:52 PM, Pamod Sylvester <[email protected]> wrote: >>> >>> Hi All, >>> >>> Since QoS 0 in MQTT is a performance oriented delivery mode, an >>> in-memory based message store was implemented to bypass the message >>> database when running MB in stand alone mode, however when running in >>> cluster in-memory store will not be applicable, since in order to exchange >>> messages across the cluster, it requires a mechanism to communicate between >>> brokers (Which will be introduced in future MB versions) >>> >>> In Stand-Alone mode the current message flow would be similar to the >>> following, >>> >>> Publisher ---> Message ---> Deliver from In-Memory Store ---> Deliver >>> from Andes Kernal Store ---> Message ---> Subscription >>> >>> In the above flow, the subscriptions with QoS 0 would receive the >>> messages through the in-memory store and the also currently the published >>> message would be sent to the Andes kernal as well. Since for the same topic >>> there could be subscriptions which are > QoS 0. Making a call to Andes >>> Kernal strore would not make much sense if there're no subscriptions for a >>> particular topic which is > QoS 0, also it will cause additional operation >>> overhead. Therefore, we're in the process of implementing a solution which >>> would be able to identify whether there are subscriptions > QoS 0 for a >>> given topic to identify the necessity to perform a call to the store. >>> >>> If we maintain the state of topics which has subscriptions > QoS 0, for >>> each message published it would require to iterate through all the >>> subscriptions O(n), since there will be cases where the published topic >>> will not 1:1 map with the subscribed topic, (i.e hierarchical topics with >>> wildcards). In order to minimize the iteration cycles one option would be >>> to partition subscriptions based on its attributes such as hierarchy, >>> inclusive of wild card characters etc so that we could omit odds and >>> iterate only through the possibilities. >>> >>> WDYT ? Please do advice on most optimized ways of filtering out matching >>> subscriptions. >>> >>> Thanks, >>> Pamod >>> >>> -- >>> *Pamod Sylvester * >>> *Senior Software Engineer * >>> Integration Technologies Team, WSO2 Inc.; http://wso2.com >>> email: [email protected] cell: +94 77 7779495 >>> >> >> >> -- >> *Asitha Nanayakkara* >> Software Engineer >> WSO2, Inc. http://wso2.com/ >> Mob: + 94 77 85 30 682 >> >> > > -- > *Pamod Sylvester * > *Senior Software Engineer * > Integration Technologies Team, WSO2 Inc.; http://wso2.com > email: [email protected] cell: +94 77 7779495 >
-- Ramith Jayasinghe Technical Lead WSO2 Inc., http://wso2.com lean.enterprise.middleware E: [email protected] P: +94 777542851
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
