On Fri, Jun 28, 2013 at 2:39 PM, Viral Bajaria <viral.baja...@gmail.com> wrote: > On Fri, Jun 28, 2013 at 9:31 AM, Jean-Daniel Cryans > <jdcry...@apache.org>wrote: > >> On Thu, Jun 27, 2013 at 4:27 PM, Viral Bajaria <viral.baja...@gmail.com> >> wrote: >> It's not random, it picks the region with the most data in its memstores. >> > > That's weird, because I see some of my regions which receive the least > amount of data in a given time period flushing before the regions that are > receiving data continuously. The reason I know this is because of the write > pattern. Some of my tables are in catch-up mode i.e. I am ingesting data > from the past and they always have something to do. While some tables are > not in catch-up mode and are just sitting idle for most of the time. Yet I > see high number of flushes for those regions too. > > >> >> I doubt that it's the fact that it's a major compaction that it's >> making everything worse. When a minor gets promoted into a major it's >> because we're already going to compact all the files, so we might as >> well get rid of some deletes at the same time. They are all getting >> selected because the files are within the selection ratio. I would not >> focus on this to resolve your problem. >> > > I meant worse for my writes not for HBase as a whole. > > >> >> I haven't been closely following this thread, but have you posted a >> log snippet somewhere? It's usually much more telling and we eliminate >> a few levels of interpretation. Make sure it's at DEBUG, and that you >> grab a few hours of activity. Get the GC log for the same time as >> well. Drop this on a web server or pastebin if it fits. >> > > The only log snippet that I posted was the flushing action. Also that log > was not everything, I had grep'd a few lines out. Let me collect some more > stats here and post it again. I just enabled GC logging on this server, > deployed the wrong config out initially which had no GC logging. I am not > sure how GC logs will help here given that I am at less than 50% heap space > used and so I would doubt a stop the world GC happening. Are you trying to > look for some other information ?
Just trying to cover all the bases. J-D
