Thanks Sinthuja. Is there any performance/operational level issue if we run this archive script every day in a production environment?
Regards, Chanaka On Mon, Jan 6, 2014 at 11:51 PM, Sinthuja Ragendran <[email protected]>wrote: > Hi, > > > On Tue, Jan 7, 2014 at 9:49 AM, Chanaka Fernando <[email protected]>wrote: > >> Hi Sinthuja, >> >> In that case, can we purge the data which is older than today for every >> hive job execution? For example, let's say I have configured the archive >> job to be run everyday 11.00 p.m. When the hive script runs, will it purge >> the data that is older than today (tomorrow and all the data which are >> older)? >> > > I think you mean, purging yesterday and all the data older than yesterday. > if you are not concerned about the time and only the day, then it's > possible. > > FYI [1] is the hive UDF which basically finds the date and the data which > is older then this date will be purged. Also since it runs in each > iteration of the hive job, UDF evaluated each time, and remove the older > data as you specify. > > [1] > https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/components/bam2/org.wso2.carbon.bam.cassandra.data.archive/4.2.0/src/main/java/org/wso2/carbon/bam/cassandra/data/archive/udf/GetPastDate.java > > Thanks, > Sinthuja. > >> >> > If this is possible with the current version, that will be enough for the >> requirement. >> Please confirm. >> >> >> Thanks, >> Chanaka >> >> >> On Mon, Jan 6, 2014 at 11:12 PM, Sinthuja Ragendran <[email protected]>wrote: >> >>> Hi, >>> >>> Yes, as per now it'll it's not purging the data which is older than >>> given time; Ex: let say yesterday 12:00 noon. It'll only consider the date, >>> not the time; ie, it'll purge the data which is older than yesterday >>> 00:00. Basically in each scheduled purging hive job execution, it's >>> obtaining the current time stamp, resetting to the beginning of the day and >>> deducting the the number of days to find out the the data that should be >>> removed. Hence it won't consider the time, rather it'll only consider the >>> current day. >>> >>> But of course we can fix this, you can report a JIRA issue and we will >>> fix it in the next release. >>> >>> Thanks, >>> Sinthuja. >>> >>> >>> On Tue, Jan 7, 2014 at 9:07 AM, Paul Fremantle <[email protected]> wrote: >>> >>>> My understanding is that it CAN but it does not, unless this is changed >>>> in 240. We definitely need to make sure that the default BAM analytics are >>>> doing some purging. >>>> >>>> Paul >>>> >>>> >>>> On Tuesday, January 7, 2014, Chanaka Fernando wrote: >>>> >>>>> Hi All, >>>>> >>>>> According to the BAM documentation [1], We can setup the BAM to >>>>> archive data for a specific time period or for a given number of days by >>>>> running a cron expression. If configure the number of days and the cron >>>>> expression, Does BAM purge the data which is older than the given number >>>>> of >>>>> days (ex: 90 days)? >>>>> >>>>> [1] http://docs.wso2.org/display/BAM240/Archive+Cassandra+Data >>>>> >>>>> Thanks, >>>>> Chanaka >>>>> >>>>> -- >>>>> -- >>>>> Chanaka Fernando >>>>> Technical Lead >>>>> WSO2, Inc.; http://wso2.com >>>>> lean.enterprise.middleware >>>>> >>>>> mobile: +94 773337238 >>>>> Blog : http://soatutorials.blogspot.com >>>>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 >>>>> Twitter:https://twitter.com/chanakaudaya >>>>> Wordpress:http://chanakaudaya.wordpress.com >>>>> >>>>> >>>>> >>>>> >>>> >>>> -- >>>> Paul Fremantle >>>> CTO and Co-Founder, WSO2 >>>> OASIS WS-RX TC Co-chair, Apache Member >>>> >>>> UK: +44 207 096 0336 >>>> US: +1 646 595 7614 >>>> >>>> blog: http://pzf.fremantle.org >>>> twitter.com/pzfreo >>>> [email protected] >>>> >>>> wso2.com Lean Enterprise Middleware >>>> >>>> Disclaimer: This communication may contain privileged or other >>>> confidential information and is intended exclusively for the addressee/s. >>>> If you are not the intended recipient/s, or believe that you may have >>>> received this communication in error, please reply to the sender indicating >>>> that fact and delete the copy you received and in addition, you should not >>>> print, copy, retransmit, disseminate, or otherwise use the information >>>> contained in this communication. Internet communications cannot be >>>> guaranteed to be timely, secure, error or virus-free. The sender does not >>>> accept liability for any errors or omissions. >>>> >>>> >>> >>> >>> -- >>> *Sinthuja Rajendran* >>> Software Engineer <http://wso2.com/> >>> WSO2, Inc.:http://wso2.com >>> >>> Blog: http://sinthu-rajan.blogspot.com/ >>> Mobile: +94774273955 >>> >>> >>> >> >> >> -- >> -- >> Chanaka Fernando >> Technical Lead >> WSO2, Inc.; http://wso2.com >> lean.enterprise.middleware >> >> mobile: +94 773337238 >> Blog : http://soatutorials.blogspot.com >> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 >> Twitter:https://twitter.com/chanakaudaya >> Wordpress:http://chanakaudaya.wordpress.com >> >> >> >> > > > -- > *Sinthuja Rajendran* > Software Engineer <http://wso2.com/> > WSO2, Inc.:http://wso2.com > > Blog: http://sinthu-rajan.blogspot.com/ > Mobile: +94774273955 > > > -- -- Chanaka Fernando Technical Lead WSO2, Inc.; http://wso2.com lean.enterprise.middleware mobile: +94 773337238 Blog : http://soatutorials.blogspot.com LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 Twitter:https://twitter.com/chanakaudaya Wordpress:http://chanakaudaya.wordpress.com
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
