Sure, let's discuss and see how we could incorporate this algorithm, This could also be added into the Andes Kernal as well.
Thanks, Pamod On Tue, Dec 16, 2014 at 7:54 AM, Asitha Nanayakkara <[email protected]> wrote: > > Hi Pamod, > > Inverted bitmap technique[1] looks good for the purpose. Algorithm is O(n) > where n is number of sections in the destination, not subscribers. > > Only concern is memory consumption and updating the maps when new > subscribers come while messages are flowing through the system at higher > rates (This will introduce some pauses to delivery). Since adding and > removing subscriptions doesn't happen frequently we can live with that I > think. Need to look at the memory consumption to make sure this is > feasible. > > [1] http://zeromq.org/whitepapers:message-matching > > On Mon, Dec 15, 2014 at 5:52 PM, Pamod Sylvester <[email protected]> wrote: >> >> Hi All, >> >> Since QoS 0 in MQTT is a performance oriented delivery mode, an in-memory >> based message store was implemented to bypass the message database when >> running MB in stand alone mode, however when running in cluster in-memory >> store will not be applicable, since in order to exchange messages across >> the cluster, it requires a mechanism to communicate between brokers (Which >> will be introduced in future MB versions) >> >> In Stand-Alone mode the current message flow would be similar to the >> following, >> >> Publisher ---> Message ---> Deliver from In-Memory Store ---> Deliver >> from Andes Kernal Store ---> Message ---> Subscription >> >> In the above flow, the subscriptions with QoS 0 would receive the >> messages through the in-memory store and the also currently the published >> message would be sent to the Andes kernal as well. Since for the same topic >> there could be subscriptions which are > QoS 0. Making a call to Andes >> Kernal strore would not make much sense if there're no subscriptions for a >> particular topic which is > QoS 0, also it will cause additional operation >> overhead. Therefore, we're in the process of implementing a solution which >> would be able to identify whether there are subscriptions > QoS 0 for a >> given topic to identify the necessity to perform a call to the store. >> >> If we maintain the state of topics which has subscriptions > QoS 0, for >> each message published it would require to iterate through all the >> subscriptions O(n), since there will be cases where the published topic >> will not 1:1 map with the subscribed topic, (i.e hierarchical topics with >> wildcards). In order to minimize the iteration cycles one option would be >> to partition subscriptions based on its attributes such as hierarchy, >> inclusive of wild card characters etc so that we could omit odds and >> iterate only through the possibilities. >> >> WDYT ? Please do advice on most optimized ways of filtering out matching >> subscriptions. >> >> Thanks, >> Pamod >> >> -- >> *Pamod Sylvester * >> *Senior Software Engineer * >> Integration Technologies Team, WSO2 Inc.; http://wso2.com >> email: [email protected] cell: +94 77 7779495 >> > > > -- > *Asitha Nanayakkara* > Software Engineer > WSO2, Inc. http://wso2.com/ > Mob: + 94 77 85 30 682 > > -- *Pamod Sylvester * *Senior Software Engineer * Integration Technologies Team, WSO2 Inc.; http://wso2.com email: [email protected] cell: +94 77 7779495
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
