Hi Chanaka, AFAIK there are no issues except one. That is tombstone issue [1][2] (Actually not an issue. That's the way Cassandra behaves). Normally when you delete records from Cassandra those records turn in to tombstones until the specified time period expires. Default values is 10 days. At the moment this value can't configure through BAM. So you have to configure this value through cassandra-cli tool against each column family.
[1] - http://wiki.apache.org/cassandra/DistributedDeletes [2] - http://en.wikipedia.org/wiki/Tombstone_(data_store) Regards, Gihan On Tue, Jan 7, 2014 at 10:25 AM, Chanaka Fernando <[email protected]> wrote: > Thanks Sinthuja. Is there any performance/operational level issue if we > run this archive script every day in a production environment? > > Regards, > Chanaka > > > On Mon, Jan 6, 2014 at 11:51 PM, Sinthuja Ragendran <[email protected]>wrote: > >> Hi, >> >> >> On Tue, Jan 7, 2014 at 9:49 AM, Chanaka Fernando <[email protected]>wrote: >> >>> Hi Sinthuja, >>> >>> In that case, can we purge the data which is older than today for every >>> hive job execution? For example, let's say I have configured the archive >>> job to be run everyday 11.00 p.m. When the hive script runs, will it purge >>> the data that is older than today (tomorrow and all the data which are >>> older)? >>> >> >> I think you mean, purging yesterday and all the data older than >> yesterday. if you are not concerned about the time and only the day, then >> it's possible. >> >> FYI [1] is the hive UDF which basically finds the date and the data which >> is older then this date will be purged. Also since it runs in each >> iteration of the hive job, UDF evaluated each time, and remove the older >> data as you specify. >> >> [1] >> https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/components/bam2/org.wso2.carbon.bam.cassandra.data.archive/4.2.0/src/main/java/org/wso2/carbon/bam/cassandra/data/archive/udf/GetPastDate.java >> >> Thanks, >> Sinthuja. >> >>> >>> >> If this is possible with the current version, that will be enough for the >>> requirement. >>> Please confirm. >>> >>> >>> Thanks, >>> Chanaka >>> >>> >>> On Mon, Jan 6, 2014 at 11:12 PM, Sinthuja Ragendran >>> <[email protected]>wrote: >>> >>>> Hi, >>>> >>>> Yes, as per now it'll it's not purging the data which is older than >>>> given time; Ex: let say yesterday 12:00 noon. It'll only consider the date, >>>> not the time; ie, it'll purge the data which is older than yesterday >>>> 00:00. Basically in each scheduled purging hive job execution, it's >>>> obtaining the current time stamp, resetting to the beginning of the day and >>>> deducting the the number of days to find out the the data that should be >>>> removed. Hence it won't consider the time, rather it'll only consider the >>>> current day. >>>> >>>> But of course we can fix this, you can report a JIRA issue and we will >>>> fix it in the next release. >>>> >>>> Thanks, >>>> Sinthuja. >>>> >>>> >>>> On Tue, Jan 7, 2014 at 9:07 AM, Paul Fremantle <[email protected]> wrote: >>>> >>>>> My understanding is that it CAN but it does not, unless this is >>>>> changed in 240. We definitely need to make sure that the default BAM >>>>> analytics are doing some purging. >>>>> >>>>> Paul >>>>> >>>>> >>>>> On Tuesday, January 7, 2014, Chanaka Fernando wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> According to the BAM documentation [1], We can setup the BAM to >>>>>> archive data for a specific time period or for a given number of days by >>>>>> running a cron expression. If configure the number of days and the cron >>>>>> expression, Does BAM purge the data which is older than the given number >>>>>> of >>>>>> days (ex: 90 days)? >>>>>> >>>>>> [1] http://docs.wso2.org/display/BAM240/Archive+Cassandra+Data >>>>>> >>>>>> Thanks, >>>>>> Chanaka >>>>>> >>>>>> -- >>>>>> -- >>>>>> Chanaka Fernando >>>>>> Technical Lead >>>>>> WSO2, Inc.; http://wso2.com >>>>>> lean.enterprise.middleware >>>>>> >>>>>> mobile: +94 773337238 >>>>>> Blog : http://soatutorials.blogspot.com >>>>>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 >>>>>> Twitter:https://twitter.com/chanakaudaya >>>>>> Wordpress:http://chanakaudaya.wordpress.com >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> -- >>>>> Paul Fremantle >>>>> CTO and Co-Founder, WSO2 >>>>> OASIS WS-RX TC Co-chair, Apache Member >>>>> >>>>> UK: +44 207 096 0336 >>>>> US: +1 646 595 7614 >>>>> >>>>> blog: http://pzf.fremantle.org >>>>> twitter.com/pzfreo >>>>> [email protected] >>>>> >>>>> wso2.com Lean Enterprise Middleware >>>>> >>>>> Disclaimer: This communication may contain privileged or other >>>>> confidential information and is intended exclusively for the addressee/s. >>>>> If you are not the intended recipient/s, or believe that you may have >>>>> received this communication in error, please reply to the sender >>>>> indicating >>>>> that fact and delete the copy you received and in addition, you should not >>>>> print, copy, retransmit, disseminate, or otherwise use the information >>>>> contained in this communication. Internet communications cannot be >>>>> guaranteed to be timely, secure, error or virus-free. The sender does not >>>>> accept liability for any errors or omissions. >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Sinthuja Rajendran* >>>> Software Engineer <http://wso2.com/> >>>> WSO2, Inc.:http://wso2.com >>>> >>>> Blog: http://sinthu-rajan.blogspot.com/ >>>> Mobile: +94774273955 >>>> >>>> >>>> >>> >>> >>> -- >>> -- >>> Chanaka Fernando >>> Technical Lead >>> WSO2, Inc.; http://wso2.com >>> lean.enterprise.middleware >>> >>> mobile: +94 773337238 >>> Blog : http://soatutorials.blogspot.com >>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 >>> Twitter:https://twitter.com/chanakaudaya >>> Wordpress:http://chanakaudaya.wordpress.com >>> >>> >>> >>> >> >> >> -- >> *Sinthuja Rajendran* >> Software Engineer <http://wso2.com/> >> WSO2, Inc.:http://wso2.com >> >> Blog: http://sinthu-rajan.blogspot.com/ >> Mobile: +94774273955 >> >> >> > > > -- > -- > Chanaka Fernando > Technical Lead > WSO2, Inc.; http://wso2.com > lean.enterprise.middleware > > mobile: +94 773337238 > Blog : http://soatutorials.blogspot.com > LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 > Twitter:https://twitter.com/chanakaudaya > Wordpress:http://chanakaudaya.wordpress.com > > > > -- W.G. Gihan Anuruddha Senior Software Engineer | WSO2, Inc. M: +94772272595
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
