Hello friends,
I was using a hadoop cluster of apache hadoop (version 0.20.2) with apache
hbase-0.20.6(2 regionservers) and apache zookeeper 3.3.1 (cluster of 2).
I ran into problems when I replaced the apache hadoop 0.20.2 with CDH3(cloudera
hadoop). When I started HBase everything started fine
I merge issue HBASE-3744 to 0.90.2 and test it.
Find that Creating table fails when region server shutdown
Does it need try to one more times for putting Meta data?
public static void addRegionToMeta(CatalogTracker catalogTracker,
HRegionInfo regionInfo)
throws IOException {
Put put
Hello guys
I am running cloudere distribution cdh3u0 on my cluster with Pig and Hbase.
i can read data from hbase using the following pig query:
my_data = LOAD 'hbase://table1' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:1') ;dump my_data
but when i try to store data into
Hello friends,
I was using a hadoop cluster of apache hadoop (version 0.20.2) with apache
hbase-0.20.6(2 regionservers) and apache zookeeper 3.3.1 (cluster of 2).
I ran into problems when I replaced the apache hadoop 0.20.2 with
CDH3(cloudera
hadoop). When I started HBase everything started fine
Can you give more detail as to how many region servers were shutting down ?
Thanks
2011/4/25 Gaojinchao gaojinc...@huawei.com
I merge issue HBASE-3744 to 0.90.2 and test it.
Find that Creating table fails when region server shutdown
Does it need try to one more times for putting Meta data?
Hi there-
Review the HBase book too.
http://hbase.apache.org/book.html#datamodel
http://hbase.apache.org/book.html#client
http://hbase.apache.org/book.html#performance
-Original Message-
From: JohnJohnGa [mailto:johnjoh...@gmail.com]
Sent: Sunday, April 24, 2011 2:46 AM
To:
Andy,
What are the symptoms?
You also need to include in your classpath the directory where the zookeeper
config file (zoo.cfg) is located.
Yes, HBASE-3714 addresses the issue discussed here. Although, it does not
fully address the NPE in
Stack,
For the bulk loads doc at http://hbase.apache.org/bulk-loads.html (in the
Importing the prepared data using the completebulkload tool Section), what
about something along what's outlined below? This could also be included or
referenced from the documentation for
Thank you Julio. I just added the below (w/ minor qualification that
-c only needed if config. not already on CLASSPATH). Thanks for the
contrib.
St.Ack
On Sun, Apr 24, 2011 at 11:17 PM, Julio Lopez jclo...@andrew.cmu.edu wrote:
Stack,
For the bulk loads doc at
Julio,
We are running CDH3U0 ( hbase 0.90.1 ) so there may be some difference if you
are running 0.90.2. Running your workaround on CDH3U0 I get the following:
hadoop jar hbase-0.90.1-cdh3u0.jar completebulkload -c hbase-site.xml input
table
usage: completebulkload
What exception do you see?
Please upgrade to 0.90.2 HBase.
Thanks,
St.Ack
On Mon, Apr 25, 2011 at 2:44 AM, Rakesh Kumar Rakshit
ihavethepotent...@gmail.com wrote:
Hello friends,
I was using a hadoop cluster of apache hadoop (version 0.20.2) with apache
hbase-0.20.6(2 regionservers) and
On Sun, Apr 24, 2011 at 8:22 PM, Gan, Xiyun ganxi...@gmail.com wrote:
It works for java code, but I'm writing php scripts using thrift gateway.
What is the solution?
Hack what you need into thrift idl and regen your php bindings.
Thanks,
St.Ack
Stack:
Exception in thread pool-1-thread-9 java.lang.OutOfMemoryError: Java
heap space
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Invocation.readFields(HBaseRPC.java:120)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:959)
at
So I'm guessing that the log you pasted was from the master, and I can
see the zookeeper doing retries and strangely enough it was kicked out
by the other ZK peers:
2011-04-21 14:48:26,043 INFO org.apache.zookeeper.ClientCnxn: Socket
connection established to 162-2-77-0/162.2.77.0:2181,
Can't tell what it was because it OOME'd while reading whatever was coming in.
Did you bump the number of handlers in that cluster too? Because you
might hit what we talked about in this jira:
https://issues.apache.org/jira/browse/HBASE-3813
Chatting w/ J-D this morning, he asked if the queues
Hi Eric,
Unfortunately, the LocalJobRunner is missing a feature that is causing the
bulk load option to fail.
Are you running a MapReduce cluster? Make sure that you've configured the
jobtracker address in your mapred-site.xml if so.
-Todd
On Fri, Apr 22, 2011 at 11:09 AM, Eric Ross
thats a separate cluster, its barely getting any traffic so I don't
think queue would be an issue. We do however have very large files
stored (file per row). So question is, if this is a GET that breaks
things, how can we avoid it?
-Jack
On Mon, Apr 25, 2011 at 10:37 AM, Jean-Daniel Cryans
There's a good chance that if the region server started getting slow,
the requests from the REST servers would start piling up in the queues
and finally blow out the memory. You could confirm that by looking at
the GC logs before the OOME.
Also when it died, it should a dumped a hprof file. If
Hi,
I am trying to debug a MapReduce program and would prefer to view the debug
informations using log4j through the web gui.I tried using Log4J logger with
Commons logging and also passing the parameter
-Dhadoop.root.logger=INFO,TLA
None of these seem to show the debug information on the web
Make sure this log4j changes is on the Child processe's CLASSPATH when
it runs out on the cluster.
St.Ack
On Sun, Apr 24, 2011 at 6:42 PM, Himanish Kushary himan...@gmail.com wrote:
Hi,
I am trying to debug a MapReduce program and would prefer to view the debug
informations using log4j
Quick update:
It turns out that we needed to run bin/set_meta_memstore_size.rb (
http://hbase.apache.org/upgrading.html) . I'm curious though: I understand
that our legacy dev machine would suffer because of the old
MEMSTORE_FLUSHSIZE setting. But we setup a brand new dev box with a pristine
0.90
One region server with meta table is shutdowned.
-邮件原件-
发件人: Ted Yu [mailto:yuzhih...@gmail.com]
发送时间: 2011年4月25日 21:25
收件人: user@hbase.apache.org
主题: Re: A question about create table with regions in hbase version 0.90.3
Can you give more detail as to how many region servers were
Hi all,
We want to implement a bill query system. We have 20M users, the bill
for each user per month contains about 10 0.6K-byte records. We want
to store user bill for 6 months. Of course, user query focused on the
latest month reports. But, the user to be queried doesn't have hot
spot.
We use
Change your key to user_month.
That will put all of the records for a user together so you will only need a
single disk operation to read all of your data. Also, test the option of
putting multiple months in a single row.
On Mon, Apr 25, 2011 at 7:59 PM, Weihua JIANG
Because of your key organization you are blowing away your cache anyway so
it isn't doing you any good.
On Mon, Apr 25, 2011 at 7:59 PM, Weihua JIANG weihua.ji...@gmail.comwrote:
And we also tried to disable block cache, it seems the performance is
even a little bit better. And it we use the
Currently, to store bill records, we can achieve about 30K record/second.
Can you use bulk load? See http://hbase.apache.org/bulk-loads.html
However, the query performance is quite poor. We can only achieve
about 600~700 month_report/second. That is, each region server can
only serve query
In client code Put data into Region server has one more times:
eg：
public void processBatch(ListRow list,
for (int tries = 0; tries numRetries retry; ++tries) { // if put
data failed and try to do.
...
In function addRegionToMeta. Does it need to do this ?
My map reduce jobs, which were running fine with HBase 0.20.4 with Hadoop
0.20.2 are now failing as I try to upgrade to HBase 0.90.1 with Hadoop
0.20.2-CDH3B4.
Under 0.90.1 I see the following error,
Error initializing attempt_201104252111_0001_m_02_0:
java.io.FileNotFoundException: File
waitForMetaServerConnectionDefault() calls waitForMeta() which should have
waited for hbase.master.catalog.timeout
What's the value for hbase.master.catalog.timeout on your cluster ?
Thanks
2011/4/25 Gaojinchao gaojinc...@huawei.com
In client code Put data into Region server has one more
With CDH3B4, the hadoop processes run as separate users (like hdfs,
mapred, etc). Did you set the CDH3B4 directory permissions correctly
as described in the install document?
See: https://ccp.cloudera.com/display/CDHDOC/Upgrading+to+CDH3 and
search for permissions.
Also see this:
The query is all random read. The scenario is that a user want to
query his own monthly bill report, e.g. to query what happened on his
bill in March, or Feb, etc. Since every user may want to do so, we
can't predict who will be the next to ask for such monthly bill
report.
2011/4/26 Stack
So, you mean I shall disable block cache and make all query directly to DFS?
Then, the query latency maybe high.
And how much block cache hit ratio is considered to be acceptable? I
mean, above such ratio, block cache is benefical.
2011/4/26 Ted Dunning tdunn...@maprtech.com:
Because of your
I use two machines (each with 30 threads) to act as clients. Both
servers and clients are connected via giganet.
Thanks
Weihua
2011/4/26 Chris Tarnas c...@tarnas.org:
For your query tests, are they all from a single thread? Have you tried
reading from multiple threads/processes in parallel -
33 matches
Mail list logo
