On Fri, Jun 28, 2013 at 9:31 AM, Jean-Daniel Cryans <jdcry...@apache.org>wrote:
> On Thu, Jun 27, 2013 at 4:27 PM, Viral Bajaria <viral.baja...@gmail.com> > wrote: > It's not random, it picks the region with the most data in its memstores. > That's weird, because I see some of my regions which receive the least amount of data in a given time period flushing before the regions that are receiving data continuously. The reason I know this is because of the write pattern. Some of my tables are in catch-up mode i.e. I am ingesting data from the past and they always have something to do. While some tables are not in catch-up mode and are just sitting idle for most of the time. Yet I see high number of flushes for those regions too. > > I doubt that it's the fact that it's a major compaction that it's > making everything worse. When a minor gets promoted into a major it's > because we're already going to compact all the files, so we might as > well get rid of some deletes at the same time. They are all getting > selected because the files are within the selection ratio. I would not > focus on this to resolve your problem. > I meant worse for my writes not for HBase as a whole. > > I haven't been closely following this thread, but have you posted a > log snippet somewhere? It's usually much more telling and we eliminate > a few levels of interpretation. Make sure it's at DEBUG, and that you > grab a few hours of activity. Get the GC log for the same time as > well. Drop this on a web server or pastebin if it fits. > The only log snippet that I posted was the flushing action. Also that log was not everything, I had grep'd a few lines out. Let me collect some more stats here and post it again. I just enabled GC logging on this server, deployed the wrong config out initially which had no GC logging. I am not sure how GC logs will help here given that I am at less than 50% heap space used and so I would doubt a stop the world GC happening. Are you trying to look for some other information ? > Thx, > > J-D Thanks to everyone on this list for their time! -Viral
