If you are looking for write throughput and running on a VM you could likely have IO issues with your virtual disks.. Best practices are to put the write ahead log on a separate disk from the data folder(s). Not sure if you have done this or what physical setup you have under the VM but I would also examine your IO while you are doing this. Is there other load on the system either read or write while this is happening?
-Thunder > On Feb 16, 2014, at 8:04 PM, Erick Ramirez <er...@ramirez.com.au> wrote: > > Jacob, > > You are right in that increasing the timeout to 20,000ms (20 seconds) is a > real concern as it just hides an underlying issue with your environment. > Without additional information, I was suspecting that this could be due to > the environment not being optimised. > > These write timeouts can occur when the systems are under load or low on > resources. My questioning around memory is leading to the fact that your > system(s) may possibly be under load due to GC which points to JVM running > out of memory. > > Have a look at the logs as they will give you clues as to what is happening, > and possibly the cause of the issue. And keep us posted. Thanks! > > Cheers, > Erick > > > >> On Mon, Feb 17, 2014 at 1:41 PM, Jacob Rhoden <jacob.rho...@me.com> wrote: >> Hi Erick, >> >>> On 17 Feb 2014, at 1:19 pm, Erick Ramirez <er...@ramirez.com.au> wrote: >>> Are you able to post log snippets around the time that the timeouts occur? >>> >>> I have a suspicion you may be running out of heap memory and might need to >>> tune your environment. The INFO entries in the log should indicate this. >> >> Im kicking off the load and not watching it so I don’t have a timestamp to >> see where it occurred. After some mucking around I worked out that adding an >> extra zero to the following parameter on both nodes makes the problem has >> gone away: >> >> write_request_timeout_in_ms: 20000 >> >> Whatever that parameter exactly controls, Im pretty sure I don’t want to >> keep a 20s write timeout :D but it allows my bulk loads to run for the time >> being. >> >> The nodes are running on some test VM’s with xmx/xms set at 1Gb. So are you >> assuming that bulk counter row adding/incrementing can cause memory issues? >> How much memory do you need to allocate before this category of problem >> would disappear? >> >> Thanks, >> Jacob >
