It should, we do a detection via reflection if HDFS-200 is in place
and if not then we don't call syncFs. Don't mix a append-enabled HDFS
client with an older HDFS tho, the RPC version won't match. This means
that you have to replace the hadoop jar in HBase's lib directory.
J-D
On Mon, Aug 23, 20
Matthew,
Yes, the master won't start until the log splitting is done. By the
looks of it (such a high number of logs when the max is 32 until the
region server force flushes hlogs), it seems that your region servers
in a previous run weren't able to ensure that each HLog had 3 replicas
even tho df
I don't really see the cause of the shutdown in there, it seems it was
already under way. Do you see messages starting with "We slept" and
then telling how long it slept? It should be not very far from that in
the log.
J-D
2010/8/23 xiujin yang :
>
> Hi,
>
> RS of HBase was frequently down when r
The last log to look at would be the .out file.
J-D
2010/8/23 xiujin yang :
>
> Thank you J-D,
>
> I posted today's whole RS log:
> http://pastebin.com/djGnNJxk
>
> GC log:
> http://pastebin.com/AQH5kUCE
>
> I don't see the messages started with "We slept".
>
>
>
>
>> Date: Mon, 23 Aug 2010 23:00
.META. has a special comparator, if you specify a start row for
example you need to write it like "tablename,," because this is what
the comparator expects.
J-D
On Tue, Aug 24, 2010 at 4:39 PM, Vidhyashankar Venkataraman
wrote:
> I was trying to scan the META table of a hbase instance.. And it i
That's https://issues.apache.org/jira/browse/HBASE-2797, please
upgrade to 0.20.6 (no migration needed, just copy over the configs).
J-D
2010/8/24 xiujin yang :
>
> Thank you J-D.
>
> The out file is like this. It has an "NullPointerException" error.
>
> 2010-08-24 02:30:14.187::INFO: Logging to
Check the documentation:
http://hbase.apache.org/docs/r0.20.6/api/org/apache/hadoop/hbase/client/package-summary.html#overview
J-D
On Thu, Aug 26, 2010 at 12:41 PM, Shuja Rehman wrote:
> Hi All
>
> I am new to hbase client API and want to know how to get data from hbase
> from cleint/remote mac
HBase needs to know about the job tracker, it could be on the same
machine or distant, and that's taken care by giving HBase mapred's
configurations. Here's the relevant documentation :
http://hbase.apache.org/docs/r0.20.6/api/org/apache/hadoop/hbase/mapreduce/package-summary.html#classpath
J-D
2
It means that there's a zoo.cfg file somewhere on HBase's classpath
and that it doesn't contain the right configs (it takes precedence
over the xml configurations). Simply remove the file or remove it from
the classpath.
BTW which version are you using?
J-D
On Fri, Aug 27, 2010 at 2:49 AM, Shuja
specifc, and unrelated to how Apache HBase is shipped.
It's not the first time I see a CDH3 user with this issue.
J-D
On Fri, Aug 27, 2010 at 10:30 AM, Shuja Rehman wrote:
> Hi
> I am using Cloudera Latest Distribution. Can u let me where this file could
> be?
>
> Thanks
>
HBase with respect to querying should be treated as a (augmented)
key/value store, so it doesn't support ad hoc queries.
The closest this translates to in HBase would be for you to scan your
table for all values between row key 201002 and 201008312359
and then filter on the month, calculat
Tim,
The bulk of the documentation is here:
http://hbase.apache.org/docs/r0.89.20100726/metrics.html
HBase uses Hadoop Metrics in order to publish its metrics, which in
turn can be fed to Ganglia.
J-D
On Mon, Aug 30, 2010 at 10:25 AM, Time Less wrote:
>> Like Stack said, it's probably that its
Vidhya,
Thanks for keeping it up :)
If you trace all the references to 003404803994 in the region
server, what do you see? I think that most of the time is spent
opening the region sequentially, it'd be nice to confirm.
J-D
On Mon, Aug 30, 2010 at 10:20 AM, Vidhyashankar Venkataraman
wrote
It's more about the question missing information, like an example
output of your query and a sample of your dataset. Also you are using
0.20.2, which 4 minor revisions old.
So I tried a simple test in the shell using HBase 0.20.2 just as a sanity check:
hbase(main):005:0> scan 't'
ROW
multi-use and not easily upgraded just yet. I will work on
> setting up another test cluster with latest. Also, not in code below I am
> also using regular expression filter to not get some keys returned. Can
> setStartRow and other filters conflict?
>
> Thanks again,
>
>
It works for me, we run with 6 to 8GB. What's the error message?
J-D
On Tue, Aug 31, 2010 at 10:43 AM, Sharma, Avani wrote:
> Is there a way to specify an initial and max heap size to hbase ?
> Giving 4G to HBASE_HEAPSIZE fails to start the JVM.
>
> Thanks,
> Avani Sharma
>
>
>
It works for me, we run with 6 to 8GB. What's the error message?
J-D
On Tue, Aug 31, 2010 at 10:43 AM, Sharma, Avani wrote:
> Is there a way to specify an initial and max heap size to hbase ?
> Giving 4G to HBASE_HEAPSIZE fails to start the JVM.
>
> Thanks,
> Avani Sharma
>
>
>
One thing you can do is to kill -9 the master process, then restart it
with bin/hbase-daemon.sh start master
This will clear the master state and it will inspect the cluster when
restarting to figure where things are.
If that doesn't work you can also restart HBase completely.
Are the region ser
Very hard to tell how it got there by just looking at the end result,
but you could try using the shell tools like disable_region then
close_region, and then enable_region on
user_name_index,,1282242158507.8c9a40b89ee92e4b2f285b306a2d30ed.
Also you could giving a spin to the the latest 0.89 releas
This is errors coming from HDFS, I would start looking at the datanode
log on the same machine for any exceptions thrown at the same time.
Also make sure your cluster is properly configured according to the
last bullet point in the requirements
http://hbase.apache.org/docs/r0.20.6/api/overview-summ
Is that really a good test? Unless you are planning to write about 1TB
of new data per day into HBase I don't see how you are testing
capacity, you're more likely testing how HBase can sustain a constant
import of a lot of data. Regarding that, I'd be interested in knowing
exactly the circumstances
One trick is to pre- force flush the table. Also try out the new 0.89,
it has 2 fixes regarding a race condition between the BaseScanner and
the closing of regions. The release candidate is here
http://people.apache.org/~jdcryans/hbase-0.89.20100830-candidate-1
J-D
On Wed, Sep 1, 2010 at 11:28 AM
is hbase 0.89 version
> released
> in June 2010.
>
> Jinsong
>
>
> ------
> From: "Jean-Daniel Cryans"
> Sent: Wednesday, September 01, 2010 11:36 AM
> To:
> Subject: Re: truncate large table suggestion
>
>>
Took a quick look at your RS log, it looks like you are using a lot of
families and loading them pretty much at the same rate. Look at lines
that start with:
INFO org.apache.hadoop.hbase.regionserver.Store: Added ...
And you will see that you are dumping very small files on the
filesystem, on ave
What Stack said, and try setting your split size bigger on your tables
in order to limit the number of them. Bigger files = less smaller
files = less occupied xcievers to answer request to those files. See
the help in the shell for "alter", look for the MAX_FILESIZE value
(which is in bytes, defaul
I think Lars explains it best:
http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-log.html
Short version: writing to the WAL is a backup solution if the region
server dies, because it's the MemStore that's being used for reads
(not the WAL). If you autoFlush, then everyone can re
If you are asking about "current" solutions, then yes you can distcp
but I would consider that a last resort solution for the reasons you
described (yes, you could end up with an inconsistent state that
requires manual fixing). Also it completely bypasses row locks.
Another choice is using the Exp
Your client is trying to connect to a local zookeeper ensemble (grep
for "connectString" in the message). This means that the client
doesn't know about the proper configurations in order to connect to
the cluster. Either put your hbase-site.xml on the client's classpath
or set the proper settings o
If you have a very heavy write load (like YCSB when only inserting),
then you really have to tune HBase for that kind of workload since
it's not the "normal" use case. Setting MAX_FILESIZE really high
(1-2GB) and even pre-splitting the table when creating it (available
in 0.89) will help.
Most of
2010-09-10 11:24:05,605 535 sec: 3620416 operations; 0 current ops/sec;
> 2010-09-10 11:24:15,607 545 sec: 3648592 operations; 2817.04 current
> ops/sec; [INSERT AverageLatency(ms)=52.15]
>
> 2010/9/10 Jean-Daniel Cryans
>
>> If you have a very heavy write load (like YC
Can we see the actual line of when it died, with a lot of context and
please in a pastebin.com
Also, most of the time users get this kind of error because they
didn't configure HBase and Hadoop properly, mostly the last
requirement:
http://hbase.apache.org/docs/r0.20.6/api/overview-summary.html#r
It's usually because your scanner timed out, or because the region
moved to a new server. You can see how it's handled in
HTable.ClientScanner.next(). In any case you should see a message like
"Scanner -8711007779313115048 lease expired" in some region server,
then see when you get the exception. C
Currently merging regions can only be done while HBase is offline, a
long time ago this was opened:
https://issues.apache.org/jira/browse/HBASE-420. And some work was to
at least be able to merge regions in disabled tables:
https://issues.apache.org/jira/browse/HBASE-1621 but it requires a lot
more
level of that
> UnknownScannerException in the code from ERROR to WARN or INFO?
>
> Thanks.
>
> -Ken
>
> On Thu, Sep 16, 2010 at 9:38 AM, Jean-Daniel Cryans
> wrote:
>
>> It's usually because your scanner timed out, or because the region
>> moved to a new
It looks like your region took at long time to move (the name of the
region being dn,,1284706796459), you should grep the master log to see
why it took so long, and correlate with the region server log at
116.211.20.208.
Also please give the usual infos about your environment (versions,
hardware,
Sounds like there's an underlying HDFS issue, you should check those
machines' datanode logs at the time of the failure for any exception.
J-D
On Fri, Sep 17, 2010 at 9:14 AM, Scott Whitecross wrote:
> Hi all -
>
> A couple of nights ago I enabled cron jobs to run major compactions against
> a f
ry to restart.
>
> I have two questions:
> 1. what contract should recipient of DoNotRetryIOException follow ? On the
> surface the way TableRecordReaderImpl handles potential
> UnknownScannerException is a little confusing.
> 2. can the logic below be applied to HRegionServer.next() ?
That exception is "normal", it just means a region is either moving or
being split, unless your client has to retry 10 times and then gets a
RetriesExhaustedException. This has been discussed a few times on this
mailing list.
Also you say you got HBASE-2516, but if you really got it then you'd
see
That 127.0.0.1 is for HBase, which means that the root region from
HBase's point of view is on the local host (which makes sense because
in pseudo-distributed mode the region server runs in the same JVM as
the master).
HBase does a reverse lookup on the machine's hostname in order to get
its IP ad
Even in standalone it does need ZK's functionality, so the master runs
it inside its JVM (same for the region server). In your case it seems
like port 2181 is already taken by something else. Either change the
other service's port, or set hbase.zookeeper.property.clientPort to
2182 in conf/hbase-si
It seem that the log splitting didn't succeed (can't tell why, small
log snippet is too small). Did it get stuck on EOF or something like
that? In any case, it looks like a bug.
> * how do I avoid disruption when this sort of thing happens
Your region server was unavailable for 3 minutes, proba
Find the earliest occurrence of that error, correlate with what's in
the regionserver/master logs around that time.
J-D
On Thu, Sep 23, 2010 at 7:00 AM, Justin Cohen wrote:
> This morning my development hbase cluster stopped working. I was seeing
> this in the logs:
>
> http://pastebin.com/b1Y
It's a znode count, not machine count. It does list only 1 region server.
J-D
On Fri, Sep 24, 2010 at 3:21 AM, Andrey Timerbaev wrote:
> Hello All,
>
> In my Hadoop 0.20.1/HBase 0.20.3/Zookeeper 3.2.2 installation, which is
> running
> on only one node in a pseudo-distributed mode, I noticed, Z
99% of the time it's because there's a lot of regions in the
compaction queue and the one that needs compaction is far behind. See
this related jira https://issues.apache.org/jira/browse/HBASE-2646
One way around that problem is setting hbase.hstore.blockingStoreFiles
higher than 7, here we have i
The way it works (which is pretty rough), is that the client asks the
master to create the table and then checks the .META. table to find
out where it was assigned. If the region isn't assigned yet, the
client checks back later.
The fact that your first sleep was 2000ms long indicates that you are
We have multiple clusters (in multiple datacenters), and web serving
is separated from MR processing. We do have some jobs running on prod
but they are usually of lower intensity, run during low traffic hours,
and we limited the number of map and reduce slots to 1 each per node.
The data between t
s,
> Katharine
>
> On 9/24/2010 1:23 PM, Jean-Daniel Cryans wrote:
>>
>> The way it works (which is pretty rough), is that the client asks the
>> master to create the table and then checks the .META. table to find
>> out where it was assigned. If the region isn't
I don't know anything about that StripedHBaseTable you're using, but
when you call flushCommits on HTable it only flushes the client write
buffer, not the Memstore, so if your table was new then your data is
in that Memstore and not in a Storefile.
J-D
On Fri, Sep 24, 2010 at 3:34 PM, Ted Yu wro
> StripedHBaseTable is just a wrapper around HTable.
Ok.
> QA installed another application which queries HBase. When QA didn't see
> meaningful data from the application.
> Why didn't HBase serve data from Memstore ?
I cannot answer this question, as far as I know 0.20.6 doesn't have a
bug that
ing triggered ?
>
> Thanks
>
> On Fri, Sep 24, 2010 at 4:01 PM, Jean-Daniel Cryans
> wrote:
>
>> > StripedHBaseTable is just a wrapper around HTable.
>>
>> Ok.
>>
>> > QA installed another application which queries HBase. When QA didn't
gt;>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:512)
>>> at org.apache.hadoop.ipc.Server$Handler$1.run
Using 0.21.0 may reveal newer bugs rather than fixing your older ones.
Maybe we can help you debugging 0.20.2, what are you seeing?
J-D
On Sun, Sep 26, 2010 at 7:03 PM, Pete Tyler wrote:
> I believe my issue is within Hadoop, not HBase, and as such I was hoping to
> run with the very latest ver
> That's funny. My understanding was, region servers were redundant inherently.
> If
> they are "semiredundant", there should be a root cause like some wrong
> settings
> or a bug.
>
> Could someone from HBase experts comment on this?
0.89 is a developer release, it should be treated as such (eg
I can't grok that, can you give an example that shows actual data VS
what you're expecting?
Thx
J-D
On Mon, Sep 27, 2010 at 5:42 AM, akhilesh.kumar
wrote:
> Hi,
> I am getting less number data, when setting start and stop value for scan
> object in hbase api.The difference is in thousands, bu
bout 20 seconds ...
> very slow
>
> I'm trying to understand why I have this 20 second overhead and what I can do
> about it.
>
> My map and reduce classes are in my Hadoop classpath.
>
> On Sep 27, 2010, at 11:32 AM, Jean-Daniel Cryans wrote:
>
>> Using 0.
I'd say it's the:
2010-09-27 12:16:15,291 INFO
org.apache.hadoop.hbase.regionserver.Store: Started compaction of 943
file(s) in att of
img833,dsc03711s.jpg,1285493435306.da57612ee69d7baaefe84
eeb0e49f240. into
hdfs://namenode-rd.imageshack.us:9000/hbase/img833/da57612ee69d7baaefe84eeb0e49f240/.tm
; I wonder how it got there, I've deleted the files.
>
> -jack
>
>
> On Mon, Sep 27, 2010 at 12:42 PM, Jean-Daniel Cryans
> wrote:
>> I'd say it's the:
>>
>> 2010-09-27 12:16:15,291 INFO
>> org.apache.hadoop.hbase.regionserver.Store
:18 PM, Jack Levin wrote:
> I see it: http://pastebin.com/tgQHBSLj
>
> Interesting situation indeed. Any thoughts on how to avoid it? Have
> compaction running more aggressively?
>
> -Jack
>
> On Mon, Sep 27, 2010 at 1:00 PM, Jean-Daniel Cryans
> wrote:
>> Can
, Sep 27, 2010 at 1:46 PM, Jack Levin wrote:
> http://pastebin.com/S7ETUpSb
>
> and
>
> Too many hlogs files:
>
> http://pastebin.com/j3GMynww
>
> Why do I have so many hlogs?
>
> -Jack
>
>
> On Mon, Sep 27, 2010 at 1:33 PM, Jean-Daniel Cryans
> wrote
hem hard, also, we run varnish to store most of the files in
> frontends, so hbase here is really for file store integrity. Still
> very curios how that one region got to so many files. Perhaps there
> should be a provision to compact more aggressively?
>
> -Jack
>
>
> On Mon,
>
> Does the 0.20.6 provide the redundancy of region servers as well as the
> 0.89.20100924?
It generally does, but since HDFS 0.20 didn't support fsSync then
testing region server recovery is pretty hard (since you never know
how much data you lost). But for 0.90, which the 0.89 releases are
snap
> Will upgrading to 0.89 be a PITA?
Unless you still use the deprecated APIs, it's actually just a matter
of replacing the distribution and restarting.
>
> Should we expect to be able to upgrade the servers without losing data?
Definitely, since no upgrade of the filesystem format is required. B
ejo
wrote:
> Just a quick question that often intrigues me, why do you guys prefer the
> CDH3b2? and not a regular hadoop-0.20.X???
> Thanks in advanced.
>
>
> Renato M.
>
> 2010/9/28 Jean-Daniel Cryans
>
>> > Will upgrading to 0.89 be a PITA?
>>
>> Un
|On Wed, Sep 29, 2010 at 3:49 AM, Matthew LeMieux wrote:
> The problem I referred to was also being addressed in a separate thread,
> thanks to the contributors to the mailing list and mostly to J-D and Stack.
>
> I have recently upgraded to the 0.89.20100924 version and after more than 24
> ho
Weird indeed, even after the WAL was rolled 4 times (theoretically
256MB of data) I don't even see a flush request... although you're
running at INFO level instead of DEBUG. Could you switch that and send
us just the full log.
Thanks a lot!
J-D
On Wed, Sep 29, 2010 at 4:25 AM, Andrey Stepachev
I'd say it mostly depends on your tolerance to regions being
unavailable while the recovery happens. You have to account for the ZK
timeout (60 secs by default), plus the time to split (I don't have any
good metric for that, usually it's kinda fast but you should try it
with your data), plus the ti
Ok I found the bug, I think it's only in our distro.
Stay tuned!
J-D
On Wed, Sep 29, 2010 at 9:26 AM, Jean-Daniel Cryans wrote:
> Weird indeed, even after the WAL was rolled 4 times (theoretically
> 256MB of data) I don't even see a flush request... although you're
&g
The fix is here: http://pastebin.com/zuL23e0U
We're going to do a push to github later today, along with other
patches that require more testing.
J-D
On Wed, Sep 29, 2010 at 10:54 AM, Andrey Stepachev wrote:
> wow. i'll wait. thanks for reply.
>
> 2010/9/29 Jean-Daniel Cry
rks fine.
>
> 2010/9/29 Jean-Daniel Cryans :
>> The fix is here: http://pastebin.com/zuL23e0U
>>
>> We're going to do a push to github later today, along with other
>> patches that require more testing.
>>
>> J-D
>>
>> On Wed, Sep 2
There's no migration.
J-D
On Thu, Sep 30, 2010 at 11:24 AM, Dmitriy Lyubimov wrote:
> Hi,
>
> I tried to find the info on data migration from 0.20 to 0.89 and so far
> wasn't successful.
>
> I noticed that migrate utility is not in the code base for the 0.89 anymore.
>
>
> is there a way or howt
Sorry, no migration as in there's nothing special to do... it just
works. And like Ryan said, no coming back.
J-D
On Thu, Sep 30, 2010 at 11:30 AM, Dmitriy Lyubimov wrote:
> Are at least export file format compatible?
>
>
> On Thu, Sep 30, 2010 at 11:28 AM, Jean-Danie
Yeah that's HBASE-2915, try this release candidate instead
http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
J-D
On Thu, Sep 30, 2010 at 12:08 PM, Matt Corgan wrote:
> I'm inserting several million rows of a table that has ~50 columns, 31 of
> which are days of the month and ar
>From the brief looks of it, it seems that the master is splitting the
log files from the dead region server. It will do that while the
cluster is running and will keep answering the other region servers,
but if you restart HBase then when the master starts it will split
everything before starting
r,
>>> but the master persists in not recognizing them:
>>>
>>> 2010-10-03 18:11:35,888 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Got ZooKeeper event,
>>> state: SyncConnected, type: NodeDeleted, path: /hbase/master
>>> 2010
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
> at java.io.DataInputStream.readInt(DataInputStream.java:370)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:514)
> at
> org.apache.hadoop.hbase.ipc.HBa
I'd love to give you tips, but you didn't provide any data about the
input and output of your job, the kind of hardware you're using, etc.
At this point any suggestion would be a stab in the dark, the best I
can do is pointing to the existing documentation
http://wiki.apache.org/hadoop/PerformanceT
A row is always served a single region, so the question is more about
region distribution. In 0.20 and 0.89 you have no control over that
(the best you can do is calling close_region in the shell, where is
the region will be moved is "random"), but things are getting better
in 0.90 where you'll be
TES_WRITTEN
>
> 1,197,031,156
>
> 861,146,518
>
> 2,058,177,674
>
>
>
> Map-Reduce Framework
>
> Reduce input groups
>
> 0
>
> 597,712
>
> 597,712
>
>
>
> Combine output records
>
> 0
>
> 0
>
> 0
>
>
>
> Map input r
ce phase) - doing a Put to 3 different HBase tables
>
>
>
> -Original Message-
> From: Jean-Daniel Cryans
> To: user@hbase.apache.org
> Sent: Tue, Oct 5, 2010 11:14 pm
> Subject: Re: HBase map reduce job timing
>
>
> It'd be more useful if we knew wh
> Regarding number of map tasks 500+, 490 of them processing nothing, do you
> have an explanation
> for that?..Wondering if its kicking off too many JVMs most doing nothing..
This would mean that throughout your regions, only a few have data in
the timestamp range you're looking for.
>
> 'top'
You need to disable your table in order to alter it. Also, instead of
using admin.addColumn, simply add it directly to the HTableDescriptor
using
http://hbase.apache.org/docs/r0.20.6/api/org/apache/hadoop/hbase/HTableDescriptor.html#addFamily(org.apache.hadoop.hbase.HColumnDescriptor)
J-D
2010/1
> Also, do you think if I query using rowkey instead of hbase time stamp..it
> would not kick off that many tasks..
> since region server knows the exact locations?
I don't see how you could do that in a scalable way, unless you really
have to query a few rows (less than a million).
>
>
>
> I'v
r and then distribute the regions
>> any way you wish with new 'move' command.
>> St.Ack
>>
>> On Tue, Oct 5, 2010 at 8:13 PM, Jean-Daniel Cryans
>> wrote:
>> > A row is always served a single region, so the question is more about
>> > regi
Are you using trunk? If so, unless you'd like to help us fix bugs,
please switch to a stable release.
J-D
2010/10/6 Petrucci Andreas :
>
> Sorry J-D about the preivious double posting. I checked what you told me but
> there are new problems... I would be grateful if anyone could help me! thanks
The HBase Team would like to announce the availability of the third
in our 'developer release' series, hbase-0.89.20100924. Binary
and source tar balls are available here:
http://www.apache.org/dyn/closer.cgi/hbase/
You can browse the documentation here:
http://hbase.apache.org/docs/r0.89.201
gt;
>
> Cao
>
> On Wed, Oct 6, 2010 at 12:54 PM, Jean-Daniel Cryans
> wrote:
>
>> Like Stack said, it's available in 0.90 which we're currently wrapping
>> up (so not in 0.20.x).
>>
>> Distributing regions and distributing blocks is orthogonal (ish,
This method is synchronized, meaning that in your case another thread
is waiting on globalMemStoreLimit getting smaller by flushing. Most of
the time, this takes a very long time because the regions that are
forced flushed also have too many store files... and the requested
compaction is far behind
Not sure about the part where your table "disappears", but your
question is mostly answered in this blog post by our good friend Lars
G. http://www.larsgeorge.com/2010/05/hbase-file-locality-in-hdfs.html
J-D
On Thu, Oct 7, 2010 at 6:41 PM, yuliya Feldman wrote:
> Hello here,
>
> I have a questio
IPV6 is indeed bad, you should change your OS configuration.
Does telnet on localhost 2181 work?
J-D
On Thu, Oct 7, 2010 at 11:30 PM, aditya bhargava
wrote:
> I'm running into a bizarre issue with Thrift and Zookeeper. I'm using it
> from Ruby, and Hbase throws be an IOError each time. Here's
a
> still on data node of first region server.
>
> My replication was 1, so data belongs to just a single data node.
>
> Thanks
>
>
>
> - Original Message
> From: Jean-Daniel Cryans
> To: user@hbase.apache.org
> Sent: Fri, October 8, 2010 2:49:18 PM
>
s will be remote
>
>
>
> - Original Message ----
> From: Jean-Daniel Cryans
> To: user@hbase.apache.org
> Sent: Fri, October 8, 2010 3:28:15 PM
> Subject: Re: Question regarding regions move/assignement
>
> Even with rep=1, HDFS appears like one single filesystem to H
On Mon, Oct 11, 2010 at 4:20 AM, Andrey Stepachev wrote:
> Hi.
>
> One additional issue with column families: number of memstores. Each
> family on insert utilizies
> one memstory. If you'll write in several memstores at onces you get
> more memstores and more
> memory will be used by you region s
Section 5.1 of the Bigtable paper gives a pretty good explanation:
http://labs.google.com/papers/bigtable.html
In HBase, Chubby is replaced by ZooKeeper, root tablet by the -ROOT-
table, and METADATA tablets by the .META. table.
J-D
On Sun, Oct 10, 2010 at 10:54 PM, William Kang wrote:
> Hi,
>
No idea, the reason it died is higher in the log. Look for a message
like "Dumping metrics" and the reason should be just a few lines
higher than that.
J-D
On Sun, Oct 10, 2010 at 5:13 PM, Venkatesh wrote:
>
> Some of the region servers suddenly dying..I've pasted relevant log lines..I
> don't
> Yes. I agree. OOME unlikely. I misinterpreted my current problem.
> I found, that this (gc timeout) on my 0.89-stumpbleupon hbase occurs
> only if writeToWAL=false. My RS eats all available memory (5GB), but
> don't get OOME. I try ti figure out what is going on.
Long GC pauses happens for many
That's a lot of information to digest Tim, so bear with me if I miss
on some details :)
a) isn't really good, the big nodes have a lot of computational power
AND spindles so leaving them like that is a waste, is there's 0
locality (MR to HBase, HBase to HDFS)
b) sounds weird, would need more time
> So, even if I use get.addColumn(byte[] family, byte[] qualifier) for a
> certain cell, the HBase will have to traverse from the beginning of
> the column family to the qualifier I defined? Is it because HBase has
> to traverse all the blocks in the HFile to find the row key or the
> qualifier?
T
> I had it in my mind that HBase liked big memory, hence assuming the
> region servers should stay on the 24G machines with plenty of memory
> at their disposal. We'll come up with a test platform and then try
> some benchmarking and do a blog on it all and share.
>
They do, but because of JVM l
> If you could answer any of these
> following questions, I would greatly grateful for that.
People usually give me beer in exchange for quick help, let me know if
that works for you ;)
>
> 1. For cell size, why it should not be larger than 20m in general?
General answer: it pokes HBase in all t
ntured into Hadoop
> (http://biodivertido.blogspot.com/2008/11/reproducing-spatial-joins-using-hadoop.html)
> but might get back into it...
>
> Tim
>
> On Thu, Oct 14, 2010 at 8:07 PM, Jean-Daniel Cryans
> wrote:
>>> I had it in my mind that HBase liked big memory, hence assuming the
1 - 100 of 1546 matches
Mail list logo
