Well, I can no longer reproduce this issue on my HBase install. One
thing I did do was run major compaction on the .META. table, as it had
11 store files.
I'll keep an eye on this to see if the problem occurs again.
Thanks,
Erik
On 21 okt 2009, at 14:23, Guilherme Germoglio wrote:
Hello Erik,
I think your attachments were blocked. Could you please upload them
somewhere else?
On Wed, Oct 21, 2009 at 8:44 AM, Erik Rozendaal
<[email protected]>wrote:
Hi all,
After some performance testing on my HBase 0.20.1 development
environment
(running in pseudo- and full-distributed mode on a single laptop) I
noticed
that scanners do not get closed properly on the region server. After
creating a heap dump with Netbeans I can see the StoreScanner
instances are
still in the Store.changedReaderObservers collection.
Each StoreScanner instance has the "closed" flag set to false, so
it looks
like the StoreScanner.close() method was never called.
I double-checked my client code and counted the number of times I
create
and close a scanner, and these counts match. I also found this is
repeatable
from the hbase shell. Open the shell, scan some table, take a heap
dump and
you'll find an unclosed StoreScanner objects in the
Store.changedReaderObservers collection.
I've attached screenshots of the number of StoreScanner instances
(after
30.001 scans) and the Store.changedReaderObservers collection of
one of the
Stores (notice that the closed field's value is 0 => false).
Ultimately the region server runs out of memory and crashes.
Has anyone experience similar problems?
Regards,
Erik
--
Guilherme
msn: [email protected]
homepage: http://sites.google.com/site/germoglio/
