On Sat, Oct 9, 2010 at 8:24 PM, Jack Levin <[email protected]> wrote: > We are importing lots and lots of data, 130 TB worth. If we set > compaction limit to say 128, and blocking limit to say 200. I know we > expect longer read times unless we use a bloomfilter, however, are > they any other detrimental performance issues to be expected? Our > flush size limit is set to 100MB. I notice that REST server sometimes > is unable to talk to regions if there are too many store files, is > that expected behavior?
No. Might be something to do w/ finding a newly split region; perhaps it took a while to split? Can you find which region was problematic then grep it in master log to see what it was up around this time? During massive imports it seems that we run > into problems with REST server freezes (e.g. responds to tcp, but > neither to puts or gets), and CPU goes crazy high, and stays high > until we restart the REST server. Any ideas? > (I've attached link to jstack trace in my previous email). > There was the OOME issue. That would likely express itself as lots of CPU in the run up to the OOME. Can you find the get or put that is stalling, figure the region its intended for and again grep master logs? (I haven't looked at the jstacks -- will do so later) St.Ack
