You mean my PCs are not good enough to run HBase well ?
I've put 5 oracle tables to HBase successfully , the biggest table record
count is only 50,000.
Is there a client request limit for region server?
Two region server just serve 5 clients, it's a little strange!
Any suggestion hardware spec for HBase?
For that spec, how many clients can fetch data from HBase concurrently?
Fleming
Amandeep Khurana
<[email protected] To:
[email protected]
> cc: (bcc: Y_823910/TSMC)
Subject: Re: HBase-0.20.0 multi
read
2009/08/21 03:49
PM
Please respond to
hbase-user
On Fri, Aug 21, 2009 at 12:45 AM, <[email protected]> wrote:
>
> I have 3 PC cluster.(pc1 , pc2 , pc3)
> Hadoop master (pc1), 2 slaves (pc2,pc3)
>
> HBase and ZK running on pc1, two region servers (pc2,pc3)
>
> pc1 : Intel core2 , 2.4GHz , RAM 1G
>
> pc2 : Intel core2 , 2.4GHz , RAM 1G
>
> pc3 : Intel core2 , 1.86GHZ, RAM 2G
>
This is a very low config for HBase. I doubt if you'll be able to get a
remotely stable hbase instance going in this. More so, if you are trying to
test how much load it can take...
>
> -----------------------------------------------------------
>
> hbase-env.sh
> export HBASE_MANAGES_ZK=true
>
> -----------------------------------------------------------
> <configuration>
>
> <property>
> <name>hbase.cluster.distributed</name>
> <value>true</value>
> <description>true:fully-distributed with unmanaged Zookeeper Quorum
> </description>
> </property>
>
> <property>
> <name>hbase.rootdir</name>
> <value>hdfs://convera:9000/hbase</value>
> <description>The directory shared by region servers.
> Should be fully-qualified to include the filesystem to use.
> E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> </description>
> </property>
>
> <property>
> <name>hbase.master</name>
> <value>10.42.253.182:60000</value>
> <description>The host and port that the HBase master runs at.
> A value of 'local' runs the master and a regionserver in
> a single process.
> </description>
> </property>
> <property>
> <name>hbase.zookeeper.quorum</name>
> <value>convera</value>
> <description>Comma separated list of servers in the ZooKeeper Quorum.
> For example,
> "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
> By default this is set to localhost for local and pseudo-distributed
> modes
> of operation. For a fully-distributed setup, this should be set to a
> full
> list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
> hbase-env.sh
> this is the list of servers which we will start/stop ZooKeeper on.
> </description>
> </property>
>
> <property>
> <name>hbase.zookeeper.property.maxClientCnxns</name>
> <value>30</value>
> <description>Property from ZooKeeper's config zoo.cfg.
> Limit on number of concurrent connections (at the socket level) that a
> single client, identified by IP address, may make to a single member
of
> the ZooKeeper ensemble. Set high to avoid zk connection issues running
> standalone and pseudo-distributed.
> </description>
> </property>
>
> </configuration>
>
>
>
>
>
>
>
> Amandeep Khurana
> <[email protected] To:
> [email protected]
> > cc: (bcc:
Y_823910/TSMC)
> Subject: Re: HBase-0.20.0
> multi read
> 2009/08/21 11:54
> AM
> Please respond to
> hbase-user
>
>
>
>
>
>
> You ideally want to have 3-5 servers outside the hbase servers... 1
> server is not enough. That could to be causing you the trouble.
>
> Post logs from the master and the region server where the read failed.
>
> Also, what's your configuration? How many nodes, ram, cpus etc?
>
> On 8/20/09, [email protected] <[email protected]> wrote:
> >
> > Hi there,
> >
> > It worked well while I fired 5 threads to fetch data from HBASE,but
> > it failed after I incresed to 6 threads.
> > Although it showed some WARN, the thread job can't be done!
> > My hbase is the latest version hbase0.20.
> > I want to test HBase multi read performance.
> > Any suggestion?
> > Thank you
> >
> > Fleming
> >
> >
> > hbase-env.sh
> > export HBASE_MANAGES_ZK=true
> >
> > 09/08/21 09:54:07 WARN zookeeper.ZooKeeperWrapper: Failed to create
> /hbase
> > -- check quorum servers, currently=10.42.253.182:2181
> > org.apache.zookeeper.KeeperException$ConnectionLossException:
> > KeeperErrorCode = ConnectionLoss for /hbase
> > at
> > org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
> > at
> > org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:522)
> > at
> >
>
>
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureExists(ZooKeeperWrapper.java:342)
>
> > at
> >
>
>
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureParentExists(ZooKeeperWrapper.java:365)
>
> > at
> >
>
>
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.checkOutOfSafeMode(ZooKeeperWrapper.java:478)
>
> > at
> >
>
>
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:846)
>
> > at
> >
>
>
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:515)
>
> > at
> >
>
>
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491)
>
> > at
> >
>
>
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:565)
>
> > at
> >
>
>
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:524)
>
> > at
> >
>
>
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491)
>
> > at
> >
>
>
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:565)
>
> > at
> >
>
>
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:528)
>
> > at
> >
>
>
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491)
>
> > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:123)
> > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:101)
> > at
> > org.gridgain.examples.executor.FlowJob.getHBaseData(FlowJob.java:144)
> > at org.gridgain.examples.executor.FlowJob.call(FlowJob.java:78)
> > at org.gridgain.examples.executor.FlowJob.call(FlowJob.java:1)
> > at
> >
>
>
org.gridgain.grid.kernal.executor.GridExecutorCallableTask$1.execute(GridExecutorCallableTask.java:57)
>
> > at
> >
>
>
org.gridgain.grid.kernal.processors.job.GridJobWorker.body(GridJobWorker.java:406)
>
> > at
> >
org.gridgain.grid.util.runnable.GridRunnable$1.run(GridRunnable.java:142)
> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> > at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> > at
> > org.gridgain.grid.util.runnable.GridRunnable.run(GridRunnable.java:194)
> > at
> >
>
>
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> > at
> >
>
>
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> > at java.lang.Thread.run(Thread.java:619)
> >
>
---------------------------------------------------------------------------
> > TSMC PROPERTY
> > This email communication (and any attachments) is proprietary
> information
> > for the sole use of its
> > intended recipient. Any unauthorized review, use or distribution by
> anyone
> > other than the intended
> > recipient is strictly prohibited. If you are not the intended
> recipient,
> > please notify the sender by
> > replying to this email, and then delete this email and any copies of
it
> > immediately. Thank you.
> >
>
---------------------------------------------------------------------------
> >
> >
> >
> >
>
>
> --
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
>
>
>
>
---------------------------------------------------------------------------
> TSMC PROPERTY
> This email communication (and any attachments) is proprietary
information
> for the sole use of its
> intended recipient. Any unauthorized review, use or distribution by
anyone
> other than the intended
> recipient is strictly prohibited. If you are not the intended
recipient,
> please notify the sender by
> replying to this email, and then delete this email and any copies of it
> immediately. Thank you.
>
>
---------------------------------------------------------------------------
>
>
>
>
---------------------------------------------------------------------------
TSMC PROPERTY
This email communication (and any attachments) is proprietary information
for the sole use of its
intended recipient. Any unauthorized review, use or distribution by anyone
other than the intended
recipient is strictly prohibited. If you are not the intended recipient,
please notify the sender by
replying to this email, and then delete this email and any copies of it
immediately. Thank you.
---------------------------------------------------------------------------
