On Sun, Jul 22, 2012 at 8:36 PM, Sateesh Lakkarsu <[email protected]> wrote: > Unfortunately, I cannot upgrade to 0.92.x/cdh4 right away and have limited > hardware for some time, so want to understand the reasoning behind the > HFile limit in v1 so that I can weigh my options. (make use of bigger > region size, increase number of regions/regionserver, wait for more nodes > and spread) > > - why is the limit 4GB in 0.90.x? > > - if it is a hard limit... would like to hear experiences from people about: >
Its not a hard limit You've seen the note here: http://hbase.apache.org/book.html#d1952e10888 ? > I have pre-split regions with known bounds, lots of RAM on each node (96G), > no swap, controlled compactions, no MR on this etc... so I believe I have > optimal set-up. > Can you try your hfilev1s with big files and see what you get? Maybe if your cell size is large and your keys small, you might be able to go relatively big with hfilev1 regions? St.Ack
