Hi Sinthuja, I'll try to use 'Incremental Analysis' feature for this. Where does the streams.properties file exist? Do I need to have an input stream which writes data to cassandra, to use this feature?
Thank You! On Tue, Feb 4, 2014 at 10:20 AM, Sinthuja Ragendran <[email protected]>wrote: > Hi chamila, > > In hive you can't delete the column family, it's not supported in the > cassandra strorage handler. > > If your requirement is incrementally processing, then better option would > be using 'Incremental Analysis' feature [1], which was introduced in BAM > 2.4.0. This wasn't fully tested with external hadoop node, etc in BAM 2.4.0 > and it will be properly tested in the next BAM release 2.4.1. But it was > working well without any issue in the standalone setup in BAM 2.4.0, hence > you could use that feature in IMO for your requirement. > > [1] http://docs.wso2.org/pages/viewpage.action?pageId=32345660 > > Thanks, > Sinthuja. > > > On Mon, Feb 3, 2014 at 11:49 PM, Chamila Wijayarathna <[email protected]>wrote: > >> Hi Sinthuja, >> In our problem, our main requirement is to delete older data from >> cassandra column family to reduce execution time of hive scripts. As we >> found on internet [1] hive does not support operations like delete and >> update. That's why we thought about archiving data from cassandra. Is there >> any way we can do this through hive? >> >> 1. https://cwiki.apache.org/confluence/display/Hive/Tutorial >> >> Thank You. >> >> >> On Mon, Feb 3, 2014 at 3:54 PM, Sinthuja Ragendran <[email protected]>wrote: >> >>> Hi chamila, >>> >>> BAM Cassandra archival feature is designed for event streams, not for >>> all cassandra column family. Hence for arbitrary column family you can't >>> use this feature. CEP and BAM uses different stream definition store, hence >>> the streams defined in CEP is not accessible in BAM. And also there is a >>> limitation in BAM archival feature, that you can't archive hours old data >>> and it will handled with days granularity. But this can be fixed easily, we >>> will fix it so that you can archive with hours granularity in the next >>> release. >>> >>> Anyhow finally the archive feature is generating hive script based on >>> the stream name and the date/time you have provided in the UI and executes >>> in a scheduled time interval to archive the data from the source column >>> family. Hence you can also write similar hive script in your own which >>> reads from your original source column family and archive to another column >>> family, which does the same job as archival feature. >>> >>> Thanks, >>> Sinthuja. >>> >>> >>> On Sun, Feb 2, 2014 at 11:00 PM, Chamila Wijayarathna >>> <[email protected]>wrote: >>> >>>> Hello all, >>>> I'm writing data into cassandra column family in BAM using CEP >>>> Cassandra output adapter. >>>> I need to send old data in this column family to an archive as >>>> mentioned in [1]. >>>> What should I use as stream name for this? In BAM management console it >>>> doesn't show any event streams. I tried output event stream name I used in >>>> CEP, but seems like BAM doesn't recognize it. >>>> Also can I Archive data which are older like 4 hours or 6 hours? It >>>> seems like BAM only support to archive data older than an integer number of >>>> days. >>>> >>>> Thank You >>>> >>>> >>>> 1. http://docs.wso2.org/display/BAM240/Archive+Cassandra+Data >>>> -- >>>> *Chamila Wijayarathna* >>>> Engineering Intern, >>>> WSO2 Inc. >>>> >>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> [email protected] >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> -- >>> *Sinthuja Rajendran* >>> Software Engineer <http://wso2.com/> >>> WSO2, Inc.:http://wso2.com >>> >>> Blog: http://sinthu-rajan.blogspot.com/ >>> Mobile: +94774273955 >>> >>> >>> >> >> >> -- >> *Chamila Wijayarathna* >> Engineering Intern, >> WSO2 Inc. >> >> > > > -- > *Sinthuja Rajendran* > Software Engineer <http://wso2.com/> > WSO2, Inc.:http://wso2.com > > Blog: http://sinthu-rajan.blogspot.com/ > Mobile: +94774273955 > > > -- *Chamila Wijayarathna* Engineering Intern, WSO2 Inc.
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
