Thanks Anjana for the clarifications! On Tue, Mar 29, 2016 at 11:29 PM, Anjana Fernando <[email protected]> wrote:
> Hi Supun, > > We should index all the information from the messages, including the data > payload as well, where this data will be very valuable in the search > operations later. This will not become that much of a big problem, since > the indexing happens in the background, and is not in the data publishing > critical path. And not to mention, the indexing is also pretty fast. Also, > disk space is a different concern, and most probably is not a big problem, > and for a very large number of messages throughout a large period of time, > if needed, we can of course configure data purging. And also, yeah, when > data is purging, it will also delete the corresponding index entries as > well, so the raw data in the database, and also the index data as well will > be deleted. > > Cheers, > Anjana. > > On Tue, Mar 29, 2016 at 10:48 PM, Supun Sethunga <[email protected]> wrote: > >> Hi all, >> >> As came-up during previous meetings, there was a requirement to do >> $subject, when we list down the messages come to a Proxy/API/etc.. in a >> given time range. If we are to allow $subject by *payload*, we need to >> index the payloads, at esb-analytics (DAS) side. And understandably, since >> payloads can be pretty large, indexing payloads for each and every message >> would eat-up the disk space (and any possible performance overhead? may be >> Gimantha can fill in..). >> >> Thus, are we going ahead with searching the messages by payloads, or do >> we have any alternative search mechanism? >> >> Also, would like to know, when we are purging the raw data in DAS (say, >> per-event-data came few months back), does it also clear any indexed >> information as well, which originated from the purged data? >> >> Regards, >> Supun >> -- >> *Supun Sethunga* >> Software Engineer >> WSO2, Inc. >> http://wso2.com/ >> lean | enterprise | middleware >> Mobile : +94 716546324 >> > > > > -- > *Anjana Fernando* > Senior Technical Lead > WSO2 Inc. | http://wso2.com > lean . enterprise . middleware > -- *Supun Sethunga* Software Engineer WSO2, Inc. http://wso2.com/ lean | enterprise | middleware Mobile : +94 716546324
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
