That might be a good idea but you might be able to redesign you layout of
the table
using a different key then the current one worth barnstorming.
Billy
"llpind" <[email protected]> wrote in
message news:[email protected]...
Sorry I forgot to mention the overflow then overflows into new row keys per
10,000 column entries (or some other split number).
llpind wrote:
When is the plan for releasing .20? This particular issue is really
important to us.
Stack, I also have another question: The problem we are trying to solve
doesn't really need the extra layer present in HBase (BigTable) structure
(RowResult holds row key and a HashMap of column name, value). What we
really need is a row key which simply holds a set of values. Essentially
this is a many-to-many. I wanted your thoughts on how we can go about
solving this problem (we can start another post for this if you’d like).
Is this something HBase can solve, or something that could potentially be
a HBase fork? Right now we are still in test mode, and only having to
deal with millions of columns, but in production (if the company sticks
with HBase) the columns could be in the billions. One idea we came up
with is to have an overflow table… e.g.
For a given row key we list the first 10,000 columns (values in our case),
and after that we create a column with an overflow id pointing an overflow
table which is keyed on this id.
This appears it may work, but isn’t the most elegant solution. I’d
appreciate input from anyone on this issue. Please, let me know if you
need me explain our problem in more detail.
stack-3 wrote:
On Wed, Jun 10, 2009 at 4:52 PM, llpind
<[email protected]> wrote:
Thanks. I think the problem is I have potentially millions of columns.
where a given RowResult can hold millions of columns to values. Thats
why
Map/Reduce is having problems as well (Java Heap exception). I've upped
mapred.child.java.opts, but problem presists.
See also HBASE-867: https://issues.apache.org/jira/browse/HBASE-867
St.Ack
--
View this message in context:
http://www.nabble.com/Help-with-Map-Reduce-program-tp23952252p23975432.html
Sent from the HBase User mailing list archive at Nabble.com.
