Hi,
How much heap space do you set in hbase-env.sh? How much memory do you have on your box? You may want to up the heap space for hbase if you can. HTH -Mike > Date: Mon, 22 Nov 2010 11:01:40 +0100 > Subject: LazyFetching of Row Results in MapReduce > From: [email protected] > To: [email protected] > > Hi all, > > I recently filled an hbase table with many millions of columns in each row > (!). The problem that now occured was that I always get a Heap Space Error > from the JVM with a subsequent shutdown of all regionservers in which the > error occurs. Since the error isn't thrown in any of my own classes, I think > that the problem is the following: > > * a row is always completely read into memory upon access (at least all > column families that I'm interested in) > * the Result object holds the complete family-qualifier-value pairs in a > KeyValue[] > * this is sometimes too much to be handled by the physical memory each map > can get, therefore a heap space error is thrown > > My question is now: is there any lazy fetching technique implemented within > the single key-values within one row? In my opinion it should be but I > couldn't find anything in the source code or wiki that hints to that. > > Any ideas on how to go around this problem? I had the idea to rebuild the > table schema to store more data in the row key and less data in the column > families which would make the tables "thinner" and "longer". It would work > in the current setup, however, it wouldn't solve the original problem... > > Thanks already in advance for any input on that, > > fnord999
