Yes, it was. I was dumping data from Oracle into Cassandra. On Sun, Jul 4, 2010 at 11:11 AM, Andrew Rollins <[email protected]>wrote:
> Is your IO under heavy load? If it is, that may be the cause, otherwise I'm > not sure what causes significant lag. On Linux I like to use "iostat -tx 10" > to check IO. > > - Andrew > > > On Sun, Jul 4, 2010 at 4:04 AM, David Boxenhorn <[email protected]> wrote: > >> Thank you very much! I now understand things much better. >> >> However, my configuration is as follows: >> >> <CommitLogSync>periodic</CommitLogSync> >> <CommitLogSyncPeriodInMS>10000</CommitLogSyncPeriodInMS> >> >> So I should see my commit log change after 10,000 milliseconds = 10 >> seconds? It seems to take much longer to show up. >> >> On Sun, Jul 4, 2010 at 10:52 AM, Andrew Rollins <[email protected]>wrote: >> >>> By default Cassandra syncs the commit log to disk periodically, so if you >>> are looking at file sizes, you won't see the most up to date numbers. This >>> is just like how if you tail a file that isn't flushing frequently, you >>> might wait a little while before you see the updates. >>> >>> In periodic mode, Cassandra acknowledges the write to the client >>> immediately (even before it is synced). You can run Cassandra in batch mode >>> instead, which basically means it writes in batches *and* it won't >>> acknowledge the writes to the client until it has actually synced. I'm still >>> somewhat new to this, but that's my understanding. >>> >>> Have a look at CommitLogSync in your storage-conf.xml for more info about >>> setting up syncing periods. >>> >>> As an aside, I'm not sure why the "ack immediately" or "ack after sync" >>> setting is piggybacked on the periodic vs batch setting. At first glance it >>> seems like concepts should be independent of one another. >>> >>> - Andrew >>> >>> >>> On Sun, Jul 4, 2010 at 3:34 AM, David Boxenhorn <[email protected]>wrote: >>> >>>> As I understand it, when you write to Cassandra, you are assured that, >>>> if successful, the new data has been written to a log file - so that if >>>> there is a crash your data is safe. Is this correct? >>>> >>>> If the above is correct, there is something going on that I don't >>>> understand. Are the log files to which the data is first written the ones >>>> that look like /var/lib/cassandra/commitlog/CommitLog-1277998453387.log ? >>>> The reason I ask is that when I write a lot of data, nothing seems to >>>> change >>>> in the commitlog directory for a long time, then at some point the log >>>> files >>>> in this directory get updated. It looks to me like there's memory caching >>>> involved, and the new data is not being immediately written to disk. What >>>> is >>>> going on? >>>> >>> >>> >> >
