Also, normally publisher mention whether to persist or not messages in message itself (delivery mode). So based on that MB will process messages in memory and/or persist to a persistence store. So if it's process in memory How do we communicate through the MB cluster?
http://activemq.apache.org/what-is-the-difference-between-persistent-and-non-persistent-delivery.html *Dhanuka Ranasinghe* Senior Software Engineer WSO2 Inc. ; http://wso2.com lean . enterprise . middleware phone : +94 715381915 On Mon, Jul 21, 2014 at 12:36 PM, Dhanuka Ranasinghe <[email protected]> wrote: > IMO, If we gonna keep huge messages as chunks in memory and insert into DB > as bulk it will heavily affect on MB heap memory. My suggestion is we need > to handle this case by case. For example, if it's small messages it will be > efficient to keep in memory while huge messages it will be efficient to > insert into DB early as possible and let others to use heap memory. For > this we will have to make this functionality more configurable but again we > will have to think about how gonna support fail over (probably have to > change db schema). > > *Dhanuka Ranasinghe* > > Senior Software Engineer > WSO2 Inc. ; http://wso2.com > lean . enterprise . middleware > > phone : +94 715381915 > > > On Mon, Jul 21, 2014 at 9:00 AM, Asitha Nanayakkara <[email protected]> > wrote: > >> We are planning to insert message chunks as batch insert queries. >> >> >> On Sat, Jul 19, 2014 at 11:00 AM, Dhanuka Ranasinghe <[email protected]> >> wrote: >> >>> Are we going to insert whole message or as chunks >>> On 18 Jul 2014 18:06, "Asitha Nanayakkara" <[email protected]> wrote: >>> >>>> Hi, >>>> >>>> Following is the RDBMS design for WSO2 MB 3.0.0 >>>> >>>> Messages model >>>> >>>> Message metadata model >>>> >>>> >>>> >>>> Following are the concerns came across in the discussion >>>> >>>> *- Why we use reference counting for message meta data?* >>>> >>>> Reference counting is needed to delete topic messages from the database >>>> reliably in a cluster deployment >>>> >>>> *- How to manage a large tables like Messages table?* >>>> >>>> for Messages table use database partitioning >>>> >>>> For Metadata queries there will be no SQL joins, hence even if the >>>> table would grow large that won't be an issue. >>>> >>>> Inserts and delete operation can be done as batch operations. >>>> >>>> *- Following option to save metadata was rejected due to following >>>> reasons* >>>> >>>> >>>> >>>> In the above design MB instance will create Node_Queue tables and >>>> Topic_Node_Queue >>>> tables when each node connects to a cluster. This design was rejected >>>> due to following reasons. >>>> It's DB admins tasks to create and delete tables. MB should not modify >>>> schema when joining to the cluster. There will be instances where MB users >>>> might not have privileges to create tables. >>>> >>>> *- Supporting several SQL implementations.* >>>> >>>> Since we are using simple SQL operations those will not become an >>>> issue. >>>> >>>> >>>> >>>> >>>> Thanks >>>> -- >>>> *Asitha Nanayakkara* >>>> Software Engineer >>>> WSO2, Inc. http://wso2.com/ >>>> Mob: + 94 77 85 30 682 >>>> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> *Asitha Nanayakkara* >> Software Engineer >> WSO2, Inc. http://wso2.com/ >> Mob: + 94 77 85 30 682 >> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
