Hi,
On Wed, Sep 4, 2013 at 11:36 AM, Malith Dhanushka <[email protected]> wrote: > > > > On Wed, Sep 4, 2013 at 11:22 AM, Shariq Muhammed <[email protected]> wrote: > >> On Wed, Sep 4, 2013 at 10:58 AM, Bhathiya Jayasekara >> <[email protected]>wrote: >> >>> Hi all, >>> >>> How about deleting archived data? As I remember, this requirement came >>> up recently. IMO it's good to have a archive-delete feature which can >>> be scheduled. >>> >> >> Once the data is archived to an external Cassandra ring, we should have >> the option to delete the archived data from the primary Cassandra cluster >> (where raw stats are being published), so that the new analytics job >> doesn't have to process the already analyzed / old data ... Or else time >> taken to complete map-reduce jobs will grow with the data growth ... >> > > Deleting the archived data from the primary Cassandra cluster is not > optional, once data has been archived to an external Cassandra ring that > data will be automatically removed from primary Cassandra cluster. I think > Bathiya is referring to deleting archived data which is in external > Cassandra ring and yes we can add a archive-delete option for that. > Great, and yes that's what I meant. Thanks, Bhathiya > > >>> Thanks, >>> Bhathiya >>> >>> >>> On Wed, Sep 4, 2013 at 10:18 AM, Malith Dhanushka <[email protected]>wrote: >>> >>>> Hi folks, >>>> >>>> We(BAM team, Sumedha) had a discussion about the $Subject and >>>> following are the suggested improvements for the Cassandra data archival >>>> feature in BAM. >>>> >>>> - Remove hive script based archiving and use hector API to directly >>>> issue archive queries to Cassandra (Current implementation is >>>> based on hive where it generates hive script and archiving process uses >>>> map-reduce jobs to achieve the task and it has a limitation of discarding >>>> custom key value pares in column family) >>>> >>>> - Use Task component for scheduling purposes >>>> >>>> - Archive data to external Cassandra ring >>>> >>>> - Major UI improvements >>>> - List the current archiving tasks >>>> - Edit, Remove and Schedule archiving tasks >>>> - Add new archiving task >>>> >>>> If there is any additional requirements please raise. >>>> >>>> Thanks, >>>> Malith >>>> -- >>>> Malith Dhanushka >>>> >>>> Engineer - Data Technologies >>>> *WSO2, Inc. : wso2.com* >>>> >>>> *Mobile* : +94 716 506 693 >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> *Bhathiya Jayasekara* >>> *Software Engineer,* >>> *WSO2 inc., http://wso2.com* >>> * >>> * >>> *Phone: +94715478185* >>> *LinkedIn: http://www.linkedin.com/in/bhathiyaj* >>> *Twitter: https://twitter.com/bhathiyax* >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Thanks, >> Shariq. >> Phone: +94 777 202 225 >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Malith Dhanushka > > Engineer - Data Technologies > *WSO2, Inc. : wso2.com* > > *Mobile* : +94 716 506 693 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Bhathiya Jayasekara* *Software Engineer,* *WSO2 inc., http://wso2.com* * * *Phone: +94715478185* *LinkedIn: http://www.linkedin.com/in/bhathiyaj* *Twitter: https://twitter.com/bhathiyax*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
