Are you using the cassandra mailing list? They are quite helpful. The threads you refer to are quite old, as they are incubator threads.
On Sat, Jul 21, 2012 at 12:01 AM, Harsha Kumara <[email protected]> wrote: > Hi! > > I was looked at that.Flushing is a safe way to delete the > commitogs(Draining cause flushing all columnfamilies and stop accepting > write requests to cassandra node).They still not replied to my forum post > in cassandra forum.Below two threads discuss about this matter.But only say > flushing and delete commitlogs.I suggest one way of delete commilog > files.I'm currently go through their documents if there's a another way i > ll update. > > [1] > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Why-so-many-commitlogs-tt5264497.html#a7581247 > [2] > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/1-1-not-removing-commit-log-files-td7568727.html > > > On Fri, Jul 20, 2012 at 11:07 PM, Shelan Perera <[email protected]> wrote: > >> Hi, >> >> Have you looked at this? (Which i found in wiki) >> Commit Log gets very big. Cassandra does not delete "old" commit logs. >> Why? >> >> You probably have one or more Column Families with very low throughput. >> These will typically not be flushed by crossing the throughput or >> operations thresholds, causing old commit segments to be retained until the >> memtable_flush_after_min threshold has been crossed. The default value for >> this threshold is 60 minutes and may be decreased via cassandra-cli by >> doing: >> >> update column family XXX with memtable_flush_after=YY; >> >> where YY is a number of minutes. >> >> I am not sure how safe it is to delete commit logs (Since you are doing >> it after a flush it should be fine) if it is a server shutdown >> for maintenance etc. This seems to be a real issue in Cassandra as i saw in >> few discussions. >> >> On Fri, Jul 20, 2012 at 8:45 PM, Harsha Kumara <[email protected]> wrote: >> >>> >>> >>> On Fri, Jul 20, 2012 at 2:46 PM, Tharindu Mathew <[email protected]>wrote: >>> >>>> Hi Harsha, >>>> >>>> Any luck with this? >>>> >>> They only defining a maximum size for log file.When logs filling up >>> there is maximum size for commit log.Default maximum size is >>> normally 4GB.But they not mention about automatic log file removing >>> process.They ask to flush the node and remove commitlog files in their >>> forums.So we can drain the node at the server shutdown in the OSGI bundle >>> and cleanup the commitlog directory in program(They expose a commitlog >>> directory through jmx interface and using platform mBean server we can get >>> it).It's possible. >>> >>>> >>>> On Mon, Jul 16, 2012 at 6:49 AM, Harsha Kumara <[email protected]> wrote: >>>> >>>>> >>>>> >>>>> On Sun, Jul 15, 2012 at 12:27 PM, Deependra Ariyadewa >>>>> <[email protected]>wrote: >>>>> >>>>>> >>>>>> >>>>>> On Fri, Jul 13, 2012 at 11:39 PM, Tharindu Mathew >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> This has been there for some time now. Who's looking into this? >>>>>>> >>>>>> Harsha Kumara did some research and found the flush and delete logs >>>>>> workaround. Harsh and Deep working on the issue. >>>>>> >>>>>> I go through the cassandra mailing list and they only mention about >>>>> flush and deleting commitlog files manually.We can drain the node at >>>>> carbon >>>>> server shutdown and clean the commitlog files(using program) in directory >>>>> can be done.I have posted in their forum about this i'll update if i got a >>>>> answer. >>>>> >>>>> Thank, >>>>>> >>>>>> Deependra. >>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Tharindu >>>>>>> Sent from my Android© device >>>>>>> On Jul 14, 2012 5:20 AM, "Deependra Ariyadewa" <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Jul 13, 2012 at 1:03 PM, Sinthuja Ragendran < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> On Thu, Jun 21, 2012 at 10:15 PM, Deependra Ariyadewa < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Jun 21, 2012 at 10:07 PM, Afkham Azeez <[email protected]>wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thu, Jun 21, 2012 at 10:03 PM, Deependra Ariyadewa < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Jun 21, 2012 at 9:12 PM, Afkham Azeez >>>>>>>>>>>> <[email protected]>wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Has this been sorted out? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> It looks we need operational solution for this. >>>>>>>>>>>> >>>>>>>>>>>> Cassandra creates a 128MB commit log for every restart and it >>>>>>>>>>>> is safe to delete commit logs after flush all the data in node to >>>>>>>>>>>> hard disk. >>>>>>>>>>>> >>>>>>>>>>>> Cassandra database folder in the BAM product grew due to high >>>>>>>>>>>> number of restarts which is not occurred in production systems. >>>>>>>>>>>> >>>>>>>>>>>> We do research to find more reliable solution. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Please provide a solution to this quickly. Many customers are >>>>>>>>>>> waiting to put BAM 2 into production. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> We will come up with a proper solution for this soon. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Deependra. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Deependra. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Jun 19, 2012 at 12:34 PM, Deependra Ariyadewa < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Jun 19, 2012 at 12:31 PM, Samisa Abeysinghe < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Might want to check with Cassandra community >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Jun 19, 2012 at 12:29 PM, Tharindu Mathew < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I was running a BAM server for some time (not continuously, >>>>>>>>>>>>>>>> restarting all the time), and noticed the Cassandra data has >>>>>>>>>>>>>>>> grown to over >>>>>>>>>>>>>>>> 11 GB. I did not do any load test to get this kind of data >>>>>>>>>>>>>>>> growth. It seems >>>>>>>>>>>>>>>> the commit logs are the culprit. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We need to figure out why this takes place. I have seen >>>>>>>>>>>>>>>> some new compaction options in the new Cassandra. Probably we >>>>>>>>>>>>>>>> need to >>>>>>>>>>>>>>>> enable some by default. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Deep, appreciate if you can take a look. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> This issue due to large commit log files. I will look into >>>>>>>>>>>>>> this issue. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> As mentioned before BAM pack size is growing continuously in >>>>>>>>> server restart, and goes to GB range. >>>>>>>>> >>>>>>>>> Is the solution has been found for this issue? Or else are we >>>>>>>>> going to go ahead with this behaviour? >>>>>>>>> >>>>>>>> >>>>>>>> Current solution is to flush and delete logs files. We are looking >>>>>>>> in to proper solution. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Deependra. >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Sinthuja. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Deependra. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Tharindu >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> blog: http://mackiemathew.com/ >>>>>>>>>>>>>>>> M: +94777759908 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Samisa... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Samisa Abeysinghe >>>>>>>>>>>>>>> VP Engineering >>>>>>>>>>>>>>> WSO2 Inc. >>>>>>>>>>>>>>> http://wso2.com >>>>>>>>>>>>>>> http://wso2.org >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Deependra Ariyadewa >>>>>>>>>>>>>> WSO2, Inc. http://wso2.com/ http://wso2.org >>>>>>>>>>>>>> >>>>>>>>>>>>>> email [email protected]; cell +94 71 403 5996 ; >>>>>>>>>>>>>> Blog http://risenfall.wordpress.com/ >>>>>>>>>>>>>> PGP info: KeyID: 'DC627E6F' >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>>>> email: **[email protected]* <[email protected]>* cell: +94 77 >>>>>>>>>>>>> 3320919 >>>>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>>>> twitter: >>>>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>>>> * >>>>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>>>> * >>>>>>>>>>>>> * >>>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Deependra Ariyadewa >>>>>>>>>>>> WSO2, Inc. http://wso2.com/ http://wso2.org >>>>>>>>>>>> >>>>>>>>>>>> email [email protected]; cell +94 71 403 5996 ; >>>>>>>>>>>> Blog http://risenfall.wordpress.com/ >>>>>>>>>>>> PGP info: KeyID: 'DC627E6F' >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919 >>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>> twitter: >>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>> * >>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>> * >>>>>>>>>>> * >>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Deependra Ariyadewa >>>>>>>>>> WSO2, Inc. http://wso2.com/ http://wso2.org >>>>>>>>>> >>>>>>>>>> email [email protected]; cell +94 71 403 5996 ; >>>>>>>>>> Blog http://risenfall.wordpress.com/ >>>>>>>>>> PGP info: KeyID: 'DC627E6F' >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Dev mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Deependra Ariyadewa >>>>>>>> WSO2, Inc. http://wso2.com/ http://wso2.org >>>>>>>> >>>>>>>> email [email protected]; cell +94 71 403 5996 ; >>>>>>>> Blog http://risenfall.wordpress.com/ >>>>>>>> PGP info: KeyID: 'DC627E6F' >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Dev mailing list >>>>>>>> [email protected] >>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Deependra Ariyadewa >>>>>> WSO2, Inc. http://wso2.com/ http://wso2.org >>>>>> >>>>>> email [email protected]; cell +94 71 403 5996 ; >>>>>> Blog http://risenfall.wordpress.com/ >>>>>> PGP info: KeyID: 'DC627E6F' >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> >>>> Tharindu >>>> >>>> blog: http://mackiemathew.com/ >>>> M: +94777759908 >>>> >>>> >>> >>> _______________________________________________ >>> Dev mailing list >>> [email protected] >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> *Shelan Perera* >> >> Software Engineer >> ** >> *WSO2, Inc. : wso2.com* >> lean.enterprise.middleware. >> >> *Home Page* : shelan.org >> *Blog* : blog.shelan.org >> *Linked-i*n :http://www.linkedin.com/pub/shelan-perera/a/194/465 >> *Twitter* :https://twitter.com/#!/shelan >> >> *Mobile* : +94 772 604 402 >> >> >> > -- Regards, Tharindu blog: http://mackiemathew.com/ M: +94777759908
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
