This smells of garbage and low memory. See for ref a similar problem report here - http://kr.forums.oracle.com/forums/thread.jspa?messageID=2146733
How many rest servers do you have loading all of that data? AFAIK they're stateless and loadbalancable # "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x0000000041260400 nid=0x2412 runnable # # "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x0000000041261c00 nid=0x2413 runnable # # "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00000000412b6c00 nid=0x2414 runnable -----Original Message----- From: Jack Levin [mailto:[email protected]] Sent: Sun 10/10/2010 5:24 AM To: [email protected] Subject: question about region files We are importing lots and lots of data, 130 TB worth. If we set compaction limit to say 128, and blocking limit to say 200. I know we expect longer read times unless we use a bloomfilter, however, are they any other detrimental performance issues to be expected? Our flush size limit is set to 100MB. I notice that REST server sometimes is unable to talk to regions if there are too many store files, is that expected behavior? During massive imports it seems that we run into problems with REST server freezes (e.g. responds to tcp, but neither to puts or gets), and CPU goes crazy high, and stays high until we restart the REST server. Any ideas? (I've attached link to jstack trace in my previous email). -Jack
