Hi,
On Tue, Jan 7, 2014 at 9:49 AM, Chanaka Fernando <[email protected]> wrote: > Hi Sinthuja, > > In that case, can we purge the data which is older than today for every > hive job execution? For example, let's say I have configured the archive > job to be run everyday 11.00 p.m. When the hive script runs, will it purge > the data that is older than today (tomorrow and all the data which are > older)? > I think you mean, purging yesterday and all the data older than yesterday. if you are not concerned about the time and only the day, then it's possible. FYI [1] is the hive UDF which basically finds the date and the data which is older then this date will be purged. Also since it runs in each iteration of the hive job, UDF evaluated each time, and remove the older data as you specify. [1] https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/components/bam2/org.wso2.carbon.bam.cassandra.data.archive/4.2.0/src/main/java/org/wso2/carbon/bam/cassandra/data/archive/udf/GetPastDate.java Thanks, Sinthuja. > > If this is possible with the current version, that will be enough for the > requirement. > Please confirm. > > > Thanks, > Chanaka > > > On Mon, Jan 6, 2014 at 11:12 PM, Sinthuja Ragendran <[email protected]>wrote: > >> Hi, >> >> Yes, as per now it'll it's not purging the data which is older than given >> time; Ex: let say yesterday 12:00 noon. It'll only consider the date, not >> the time; ie, it'll purge the data which is older than yesterday 00:00. >> Basically in each scheduled purging hive job execution, it's obtaining the >> current time stamp, resetting to the beginning of the day and deducting the >> the number of days to find out the the data that should be removed. Hence >> it won't consider the time, rather it'll only consider the current day. >> >> But of course we can fix this, you can report a JIRA issue and we will >> fix it in the next release. >> >> Thanks, >> Sinthuja. >> >> >> On Tue, Jan 7, 2014 at 9:07 AM, Paul Fremantle <[email protected]> wrote: >> >>> My understanding is that it CAN but it does not, unless this is changed >>> in 240. We definitely need to make sure that the default BAM analytics are >>> doing some purging. >>> >>> Paul >>> >>> >>> On Tuesday, January 7, 2014, Chanaka Fernando wrote: >>> >>>> Hi All, >>>> >>>> According to the BAM documentation [1], We can setup the BAM to archive >>>> data for a specific time period or for a given number of days by running a >>>> cron expression. If configure the number of days and the cron expression, >>>> Does BAM purge the data which is older than the given number of days (ex: >>>> 90 days)? >>>> >>>> [1] http://docs.wso2.org/display/BAM240/Archive+Cassandra+Data >>>> >>>> Thanks, >>>> Chanaka >>>> >>>> -- >>>> -- >>>> Chanaka Fernando >>>> Technical Lead >>>> WSO2, Inc.; http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> mobile: +94 773337238 >>>> Blog : http://soatutorials.blogspot.com >>>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 >>>> Twitter:https://twitter.com/chanakaudaya >>>> Wordpress:http://chanakaudaya.wordpress.com >>>> >>>> >>>> >>>> >>> >>> -- >>> Paul Fremantle >>> CTO and Co-Founder, WSO2 >>> OASIS WS-RX TC Co-chair, Apache Member >>> >>> UK: +44 207 096 0336 >>> US: +1 646 595 7614 >>> >>> blog: http://pzf.fremantle.org >>> twitter.com/pzfreo >>> [email protected] >>> >>> wso2.com Lean Enterprise Middleware >>> >>> Disclaimer: This communication may contain privileged or other >>> confidential information and is intended exclusively for the addressee/s. >>> If you are not the intended recipient/s, or believe that you may have >>> received this communication in error, please reply to the sender indicating >>> that fact and delete the copy you received and in addition, you should not >>> print, copy, retransmit, disseminate, or otherwise use the information >>> contained in this communication. Internet communications cannot be >>> guaranteed to be timely, secure, error or virus-free. The sender does not >>> accept liability for any errors or omissions. >>> >>> >> >> >> -- >> *Sinthuja Rajendran* >> Software Engineer <http://wso2.com/> >> WSO2, Inc.:http://wso2.com >> >> Blog: http://sinthu-rajan.blogspot.com/ >> Mobile: +94774273955 >> >> >> > > > -- > -- > Chanaka Fernando > Technical Lead > WSO2, Inc.; http://wso2.com > lean.enterprise.middleware > > mobile: +94 773337238 > Blog : http://soatutorials.blogspot.com > LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 > Twitter:https://twitter.com/chanakaudaya > Wordpress:http://chanakaudaya.wordpress.com > > > > -- *Sinthuja Rajendran* Software Engineer <http://wso2.com/> WSO2, Inc.:http://wso2.com Blog: http://sinthu-rajan.blogspot.com/ Mobile: +94774273955
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
