Hello Erik, I think your attachments were blocked. Could you please upload them somewhere else?
On Wed, Oct 21, 2009 at 8:44 AM, Erik Rozendaal <[email protected]>wrote: > Hi all, > > After some performance testing on my HBase 0.20.1 development environment > (running in pseudo- and full-distributed mode on a single laptop) I noticed > that scanners do not get closed properly on the region server. After > creating a heap dump with Netbeans I can see the StoreScanner instances are > still in the Store.changedReaderObservers collection. > > Each StoreScanner instance has the "closed" flag set to false, so it looks > like the StoreScanner.close() method was never called. > > I double-checked my client code and counted the number of times I create > and close a scanner, and these counts match. I also found this is repeatable > from the hbase shell. Open the shell, scan some table, take a heap dump and > you'll find an unclosed StoreScanner objects in the > Store.changedReaderObservers collection. > > I've attached screenshots of the number of StoreScanner instances (after > 30.001 scans) and the Store.changedReaderObservers collection of one of the > Stores (notice that the closed field's value is 0 => false). > > Ultimately the region server runs out of memory and crashes. > > Has anyone experience similar problems? > > Regards, > Erik > > > -- Guilherme msn: [email protected] homepage: http://sites.google.com/site/germoglio/
