Hi all, When we Allow users to have multiple table sets (for metadata and message content), each message should go to separate tables according to the queue ID. Currently I’m choosing the table for each queue in a round-robin manner (tableID = queueID % numberOfTables). Eg: If queueID is 3 and Number of tables are 5 then the table id is= (3 % 5) = 3.
For this task I need to get queueID or queueName for each method that calling METADATA_TABLE or CONTENT_TABLE. This could be achieved in several ways. 1. Get queueID or queueName from the sources and pass it through the method signatures. So need to change chains of signatures in different methods in different classes. 1. All of the methods that need to implement are currently accessing the messageID. So we can embed queueID to the messageID and pass it. Any suggestions for this approach ? Regards, Udaka On Fri, Sep 9, 2016 at 4:54 PM, Ramith Jayasinghe <[email protected]> wrote: > so the overall idea here is to figure out, having multiple tablesets to > store messages would increase the performance. This stems from the fact > that if messages for every queue writing same table set at high frequency > limits performance ( this depends on the database type, hardware sizing). > we have done performance tests for MySQL, Oracle, MSSQL. > MSSQL and Oracle gives the same level of performance (on RDS with SSD) > more or less. but MySQL is less performing. > > > On Fri, Sep 2, 2016 at 5:22 PM, Malith Jayasinghe <[email protected]> > wrote: > >> >> HI Udaka, >> >> Just wondering if has there been any performance tests (e.g. average >> query execution time/lock time using query logs etc) done on the current >> architecture under different scenarios (e.g. different number of queues >> etc)? >> >> It would be interesting to know how the overall performance degrades with >> the increasing number of queues etc and the contribution of the database to >> the overall performance degradation? >> >> thanks >> >> Malith >> >> >> On Fri, Sep 2, 2016 at 4:44 PM, Udaka Manawadu <[email protected]> wrote: >> >>> Hi All, >>> >>> Message broker uses a relational database to store messages. More >>> specifically these messages are stored in a table named as MB_CONTENT and >>> MB_METADATA. What this means is regardless of the fact that data belongs to >>> multiple queues, broker will utilize only one table-set for storage hence >>> the performance broker (cluster) suffers when number of queues grows and/or >>> cluster is in severe load. >>> >>> Figure 1: Current high-level architecture of the related tables at >>> wso2_mb. >>> >>> >>> >>> >>> >>> *Solution:* >>> >>> Allow users to have multiple table sets (for metadata and message >>> content) and/or data sources configured in message broker that will make >>> load of database is partitioned among multiple tables (or data sources) >>> thus broker’s performance is not bounded by the performance of a >>> single-table set. >>> >>> Figure 2: Suggested Solution (Number of table sets: 4) >>> >>> >>> >>> Please share your thoughts on this. >>> >>> Thanks, >>> Udaka >>> -- >>> Udaka Ayas Manawadu >>> *Intern,* >>> WSO2, Inc. >>> lean.enterprise.middleware >>> >>> Mobile: +94716836699 >>> Email: [email protected] >>> Linkedin: https://lk.linkedin.com/in/udakaayas >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Malith Jayasinghe >> >> >> WSO2, Inc. (http://wso2.com) >> Email : [email protected] >> Mobile : 0770704040 >> Lean . Enterprise . Middleware >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Ramith Jayasinghe > Technical Lead > WSO2 Inc., http://wso2.com > lean.enterprise.middleware > > E: [email protected] > P: +94 772534930 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Udaka Ayas Manawadu *Software Engineering Intern,* WSO2, Inc. lean.enterprise.middleware Mobile: +94716836699 Email: [email protected] Linkedin: https://lk.linkedin.com/in/udakaayas
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
