Hi Pamod, True it does not support triggers during expiration with TTL. I missed out on that. However, I just talked to HasithaH and he says expired active messages do not have to go to DLC, But existing messages in DLC must be checked for expiration.
Anyhow, while checking on a Cassandra talk <https://www.youtube.com/watch?v=0u-EKJBPrj8>, I came across a few more concerns regarding use of TTL columns : - Arbitrary column deletion in a single row is really not encouraged within Cassandra - since it uses SSTables , deletion requires all data in a single SSTable to be removed. resulting in deleted entries not being cleared up for a while. - TTL based deletion also falls victim to above situation though not as much. if a single row has both TTL and non-TTL columns, same issue will happen. So, *approach 1* in my mail should be done very very carefully if we have to do it. Based on your case for detecting deletions, and the fact that Cassandra could be just another message store supported by MB in future, I feel that we should go for *approach 2* ; " Implement our own storage data expiration mechanism, so that it can be re-used with any message store implementation." Will update this thread with any more findings. Others also please share your thoughts on this. Cheers, Hasitha Amal De Silva Software Engineer Mobile : 0718133463 WSO2 Inc.: http://wso2.com ( lean.enterprise.middleware. ) On Thu, Jun 19, 2014 at 9:40 PM, Pamod Sylvester <[email protected]> wrote: > Hi Hasitha, > > Does cassandra support triggers during the TTL expires ? cause according > to [1] it does not support it during the TTL. If that is the case if we > move ahead with approach (1) we will have no way of informing the MB of the > expired message, that will be a problem if we're to move the expired > messages to the DLC etc. or is it the case where the expired messages are > meant to be purged and not moved to the DLC. > > Thanks, > Pamod > > [1] > http://www.datastax.com/dev/blog/whats-new-in-cassandra-2-0-prototype-triggers-support > > > > On Thu, Jun 19, 2014 at 1:56 AM, Hasitha Amal De Silva <[email protected]> > wrote: > >> Hi all, >> >> The feature was implemented as discussed earlier (see attachment) in MB >> v3.0.0, but its yet to be improved. >> >> There is a performance concern when deleting expired messages because >> references to a message can be in many places within the Cassandra store. ( >> refer [1] >> <https://creately.com/diagram/hwkkyudn1/G7JEeNM1557U9Pm1WZEunHvdI%3D>) >> >> Trying to read and remove all occurrences of a message can slow down the >> Message expiration thread and cause a bottleneck overall. >> >> 2 approaches were discussed to resolve this : >> >> 1. Define TTL for messages whenever storing to Cassandra, so that they >> can be removed by Cassandra itself. [2] >> <http://www.datastax.com/docs/1.0/ddl/column_family> >> >> 2. Implement our own storage data expiration mechanism, so that it can be >> re-used with any message store implementation. (In Memory, NoSQL, >> relational) >> >> According to the MB team, version 3.0.0 might need to support relational >> storage as well, which is a plus point at the 2nd approach. The key >> concerns here are *re-usability* and *performance*. >> >> Please share your thoughts on how we can best handle this. >> >> [1] : https://creately.com/diagram/hwkkyudn1/G7JEeNM1557U9Pm1WZEunHvdI%3D >> [2] : http://www.datastax.com/docs/1.0/ddl/column_family >> >> >> Thanks, >> Hasitha Amal De Silva >> Software Engineer >> Mobile : 0718133463 >> WSO2 Inc.: http://wso2.com ( lean.enterprise.middleware. ) >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > *Pamod Sylvester * > *Senior Software Engineer * > Integration Technologies Team, WSO2 Inc.; http://wso2.com > email: [email protected] cell: +94 77 7779495 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
