full log is here (new) http://www.katkovonline.com/cassandra-log.zip
On Sun, Sep 27, 2009 at 12:18 PM, Igor Katkov <[email protected]> wrote: > There is no log lines with "column family 1" > There are only "column family 0" > > > On Sun, Sep 27, 2009 at 12:02 PM, Jonathan Ellis <[email protected]>wrote: > >> we're looking for "column family 1" >> >> On Sun, Sep 27, 2009 at 10:39 AM, Igor Katkov <[email protected]> wrote: >> > yes, long enough. >> > here is a typical log line from CommitLog.java >> > >> > DEBUG [COMMIT-LOG-WRITER] 2009-09-27 11:12:38,646 CommitLog.java (line >> 469) >> > discard completed log segments for >> > >> CommitLogContext(file='c:/cassandra-4rc/data/commitlog\CommitLog-1254064318865.log', >> > position=67321599), column family 0. CFIDs are BenchMarkApp: >> > TableMetadata(Channels: 0, }), system: TableMetadata(LocationInfo: 1, >> > HintsColumnFamily: 2, }), } >> > >> > >> > On Sun, Sep 27, 2009 at 8:49 AM, Jonathan Ellis <[email protected]> >> wrote: >> >> >> >> Did you let it run long enough to flush the LocationInfo CF? Should >> >> try every 5 minutes. >> >> >> >> On Sun, Sep 27, 2009 at 12:03 AM, Igor Katkov <[email protected]> >> wrote: >> >> > I applied the patch, logs segments are still not deleted >> >> > but at least with this extra logging I found something - it says "Not >> >> > safe >> >> > to delete commit log >> >> > c:/cassandra-4rc/data/commitlog\CommitLog-1254027292143.log; dirty is >> 1" >> >> > please see log snippet below, I grepped all lines that contain >> >> > "1254027292143" >> >> > full log 1Mb zipped is here >> >> > http://www.katkovonline.com/cassandra-log.zip >> >> > >> >> > >> >> > DEBUG [COMMIT-LOG-WRITER] 2009-09-27 00:54:54,817 CommitLog.java >> (line >> >> > 469) >> >> > discard completed log segments for >> >> > >> >> > >> CommitLogContext(file='c:/cassandra-4rc/data/commitlog\CommitLog-1254027292143.log', >> >> > position=708751), column family 0. CFIDs are BenchMarkApp: >> >> > TableMetadata(Channels: 0, }), system: TableMetadata(LocationInfo: 1, >> >> > HintsColumnFamily: 2, }), } >> >> > DEBUG [COMMIT-LOG-WRITER] 2009-09-27 00:54:54,817 CommitLog.java >> (line >> >> > 520) >> >> > Marking replay position on current commit log >> >> > c:/cassandra-4rc/data/commitlog\CommitLog-1254027292143.log >> >> > DEBUG [COMMIT-LOG-WRITER] 2009-09-27 00:54:58,680 CommitLog.java >> (line >> >> > 469) >> >> > discard completed log segments for >> >> > >> >> > >> CommitLogContext(file='c:/cassandra-4rc/data/commitlog\CommitLog-1254027292143.log', >> >> > position=68030159), column family 0. CFIDs are BenchMarkApp: >> >> > TableMetadata(Channels: 0, }), system: TableMetadata(LocationInfo: 1, >> >> > HintsColumnFamily: 2, }), } >> >> > DEBUG [COMMIT-LOG-WRITER] 2009-09-27 00:54:58,680 CommitLog.java >> (line >> >> > 520) >> >> > Marking replay position on current commit log >> >> > c:/cassandra-4rc/data/commitlog\CommitLog-1254027292143.log >> >> > DEBUG [COMMIT-LOG-WRITER] 2009-09-27 00:55:01,120 CommitLog.java >> (line >> >> > 540) >> >> > Not safe to delete commit log >> >> > c:/cassandra-4rc/data/commitlog\CommitLog-1254027292143.log; dirty is >> 1, >> >> > DEBUG [COMMIT-LOG-WRITER] 2009-09-27 00:55:02,715 CommitLog.java >> (line >> >> > 540) >> >> > Not safe to delete commit log >> >> > c:/cassandra-4rc/data/commitlog\CommitLog-1254027292143.log; dirty is >> 1, >> >> > DEBUG [COMMIT-LOG-WRITER] 2009-09-27 00:55:05,030 CommitLog.java >> (line >> >> > 540) >> >> > Not safe to delete commit log >> >> > c:/cassandra-4rc/data/commitlog\CommitLog-1254027292143.log; dirty is >> 1, >> >> > ... >> >> > >> >> > >> >> > P.S. >> >> > OS: Windows server 2003 >> >> > >> >> > On Sat, Sep 26, 2009 at 2:50 PM, Jonathan Ellis <[email protected]> >> >> > wrote: >> >> >> >> >> >> Patch 0001 on https://issues.apache.org/jira/browse/CASSANDRA-459 >> >> >> fixes another bug that could cause this. >> >> >> >> >> >> On Fri, Sep 25, 2009 at 11:55 PM, Igor Katkov <[email protected]> >> >> >> wrote: >> >> >> > I checked out and built 0.4 branch. It's all the same, files >> stays. >> >> >> > I also noticed a side effect - as number of commit log segments is >> >> >> > growing, server response time is also growing. >> >> >> > I assume this is because Cassandra now has to read through some >> these >> >> >> > files on reach read/write request >> >> >> > >> >> >> > >> >> >> > On Fri, Sep 25, 2009 at 4:53 PM, Jonathan Ellis < >> [email protected]> >> >> >> > wrote: >> >> >> >> This is fixed on the 0.4 branch (but not in trunk, yet) >> >> >> >> >> >> >> >> On Fri, Sep 25, 2009 at 1:57 PM, Jonathan Ellis < >> [email protected]> >> >> >> >> wrote: >> >> >> >>> https://issues.apache.org/jira/browse/CASSANDRA-455 will >> address >> >> >> >>> FlushPeriod not working. >> >> >> >>> >> >> >> >>> On Fri, Sep 25, 2009 at 1:33 PM, Igor Katkov <[email protected] >> > >> >> >> >>> wrote: >> >> >> >>>> I tried latest stable version 0.3 and commit logs segments are >> in >> >> >> >>>> fact deleted. >> >> >> >>>> Tried it again on 0.4 set periodic flush to 1min >> >> >> >>>> (FlushPeriodInMinutes="1") => it's all the same, files remains >> >> >> >>>> there >> >> >> >>>> forever. >> >> >> >>>> >> >> >> >>>> I also noticed that there are other implicit CFs, can these >> >> >> >>>> prevent >> >> >> >>>> logs from being deleted? >> >> >> >>>> DEBUG - adding Channels as 0 >> >> >> >>>> DEBUG - adding LocationInfo as 1 >> >> >> >>>> DEBUG - adding HintsColumnFamily as 2 >> >> >> >>>> >> >> >> >>>> On Thu, Sep 24, 2009 at 11:07 PM, Igor Katkov < >> [email protected]> >> >> >> >>>> wrote: >> >> >> >>>>> in my case commit log segments are never deleted (unless I >> >> >> >>>>> restart >> >> >> >>>>> the server) >> >> >> >>>>> so they grow and grow and eventually hosts is running out of >> >> >> >>>>> space. >> >> >> >>>>> >> >> >> >>>>> Any ideas how to fix it? >> >> > >> >> > >> > >> > >> > >
