parallelizing HBaseAdmin.flush()

2011-01-22 Thread Ted Yu
In 0.90, HBaseAdmin.flush() uses a loop to go over ListPairHRegionInfo, HServerAddress Should executor service be used for higher parallelism ? Thanks

cluster setup WAS: hbase config for 0.90

2011-01-22 Thread Ted Yu
Our dev cluster has 8 nodes. hadoop0: NN, JT, zookeeper quorum hadoop8: TT, HMaster hadoop4: RS All other nodes run RS and TT Is there inherent issue with this setup ? Thanks On Sat, Jan 22, 2011 at 7:44 AM, Ted Yu yuzhih...@gmail.com wrote: I encountered different kinds of failure w.r.t.

Re: HBase roadmaps

2011-01-22 Thread Lars George
Hi Stack, That's a start. Let's try it with the CP issue and try to get a report going? Lars On Jan 21, 2011, at 19:07, Stack st...@duboce.net wrote: So, add a tag VERSION_fundamental? E.g. 0.92_fundamental ? St.Ack On Fri, Jan 21, 2011 at 8:34 AM, Lars George lars.geo...@gmail.com

Re: HBase roadmaps

2011-01-22 Thread Stack
You want to try it Lars? St.Ack On Sat, Jan 22, 2011 at 9:19 AM, Lars George lars.geo...@gmail.com wrote: Hi Stack, That's a start. Let's try it with the CP issue and try to get a report going? Lars On Jan 21, 2011, at 19:07, Stack st...@duboce.net wrote: So, add a tag

Items to contribute (plan)

2011-01-22 Thread Tatsuya Kawano
Hi, I wanted to let you know that I'm planning to contribute the following items to the HBase community. These are my spare time projects and I'll only be able to spend my time about 7 hours a week, so the progress will be very slow. I want some feedback from you guys to prioritize them.

Re: Items to contribute (plan)

2011-01-22 Thread Ted Yu
#1 looks similar to what MapR has done. On Sat, Jan 22, 2011 at 5:18 PM, Tatsuya Kawano tatsuya6...@gmail.comwrote: Hi, I wanted to let you know that I'm planning to contribute the following items to the HBase community. These are my spare time projects and I'll only be able to spend my

Re: Items to contribute (plan)

2011-01-22 Thread Ryan Rawson
Hopefully to do #1, you would not require many/any changes in HFile or HBase. Implementing the HDFS stream API should be enough. #2 is interesting, what is the benefit? How did you measure said benefit? -ryan On Sat, Jan 22, 2011 at 5:45 PM, Ted Yu yuzhih...@gmail.com wrote: #1 looks similar

many 'Got brand-new compressor' messages in region server log

2011-01-22 Thread Ted Yu
In 0.90 region server log, I got: grep 'Got brand-new compressor' hbase-hadoop-regionserver-sjc1-hadoop2.sjc1.carrieriq.com.log | wc 435933 3051531 41413635 I wonder if the above led to degraded performance. On the same regionserver: libgplcompression.so libhadoop.a libhadoop.la

Re: many 'Got brand-new compressor' messages in region server log

2011-01-22 Thread Matt Corgan
That happened to me when using GZ compression without the native compression libraries installed properly. You must add them to hbase .90 manually. Without them it uses the pure-java gzip successfully, but I think hadoop has a logging bug and prints out the message too often. Matt On Sat, Jan