Hi all, With the new database design with foreign keys specified, message content get deleted automatically when message metadata is deleted. Therefore, a separate message content table is needed for messages in the DLC. And that introduces a lot of problems:
- Moving to and from DLC involves several inserts and deletions - For each message(or batch) that is read from the database, which table that message should be read from, has to be specified. Due to these reasons we decided to add just a single attribute to the metadata table indicating to which DLC queue the message was moved to. That way, we could prevent moving data from one table to another. Thank you. On Wed, Jun 17, 2015 at 2:52 PM, Sasikala Kottegoda <[email protected]> wrote: > Hi all, > > Currently WSO2MB stores message metadata in a single table with the > following attributes: > > 1. MESSAGE_ID > 2. QUEUE_ID > 3. MESSAGE_METADATA > > Also, WSO2MB supports movement of messages to a DLC (dead letter channel) > when a problem is identified in the receiver's end. When moving a message > from a queue to a DLC, the DLCQueueName (which is tenant specific) is > retrieved, the metadata row under the relevant storageQueue is removed and > a new metadata row is added with the QUEUE_ID set to the DLC queue. > > Therefore, when we need to, > > - reroute messages for a specific queue > - purge a particular queue > > we need to retrieve MESSAGE_METADATA of each row with the QUEUE_ID set to > any DLCQueue and extract the actual storageQueue (The storageQueue is one > of the information stored under the MESSAGE_METADATA attribute). This > operation is highly computationally expensive since we need to read all > rows for all DLC queues even though we might actually have a few relevant > rows. > > What I'm suggesting is to have a seperate MB_DLC_METADATA table with the > following attributes (same as the MB_METADATA table): > > 1. MESSAGE_ID > 2. SOTRAGE_QUEUE_ID > 3. MESSAGE_METADATA > > The respective cassandra column family (DLCMetaData) structure will be the > same as the current cassandra MetaData column family. Each row will have > storageQueue as the key, each message as a column with the message_id as > the column name and metadata as the value as follows: > > Queue_ID > > m_id1 > > m_id2 > > m_id3 > > meta1 > > meta2 > > meta3 > > > By having this we will be able to easily filter out messages by the > storageQueue from which the messages were moved. Therefore, it will save a > lot of time and memory in the previously mentioned scenarios and in any > other DLC specific operations. > > Comments and suggestions are highly valued. > > Thank you > > -- > Sasikala Kottegoda > *Software Engineer* > WSO2 Inc., http://wso2.com/ > lean. enterprise. middleware > Mobile: +94 774835928/712792401 > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
