On Wed, Sep 4, 2013 at 10:58 AM, Bhathiya Jayasekara <[email protected]>wrote:
> Hi all, > > How about deleting archived data? As I remember, this requirement came up > recently. IMO it's good to have a archive-delete feature which can > be scheduled. > Once the data is archived to an external Cassandra ring, we should have the option to delete the archived data from the primary Cassandra cluster (where raw stats are being published), so that the new analytics job doesn't have to process the already analyzed / old data ... Or else time taken to complete map-reduce jobs will grow with the data growth ... > > Thanks, > Bhathiya > > > On Wed, Sep 4, 2013 at 10:18 AM, Malith Dhanushka <[email protected]> wrote: > >> Hi folks, >> >> We(BAM team, Sumedha) had a discussion about the $Subject and following >> are the suggested improvements for the Cassandra data archival feature in >> BAM. >> >> - Remove hive script based archiving and use hector API to directly issue >> archive queries to Cassandra (Current implementation is based >> on hive where it generates hive script and archiving process uses >> map-reduce jobs to achieve the task and it has a limitation of discarding >> custom key value pares in column family) >> >> - Use Task component for scheduling purposes >> >> - Archive data to external Cassandra ring >> >> - Major UI improvements >> - List the current archiving tasks >> - Edit, Remove and Schedule archiving tasks >> - Add new archiving task >> >> If there is any additional requirements please raise. >> >> Thanks, >> Malith >> -- >> Malith Dhanushka >> >> Engineer - Data Technologies >> *WSO2, Inc. : wso2.com* >> >> *Mobile* : +94 716 506 693 >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > *Bhathiya Jayasekara* > *Software Engineer,* > *WSO2 inc., http://wso2.com* > * > * > *Phone: +94715478185* > *LinkedIn: http://www.linkedin.com/in/bhathiyaj* > *Twitter: https://twitter.com/bhathiyax* > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Thanks, Shariq. Phone: +94 777 202 225
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
