Hi Vimal,
This is used only with ConstantSizeRegionSplitPolicy. Which split policy do
you have configured in your setup?
JM
2014-03-06 7:55 GMT-05:00 Vimal Jain vkj...@gmail.com:
Hi,
I am running 2 node hbase cluster atop HDFS.
I was simulating a heavy write process on Hbase for
Hi Jean,
I am not sure about this.
Whats the default policy ?
On Thu, Mar 6, 2014 at 6:30 PM, Jean-Marc Spaggiari jean-m...@spaggiari.org
wrote:
Hi Vimal,
This is used only with ConstantSizeRegionSplitPolicy. Which split policy do
you have configured in your setup?
JM
2014-03-06 7:55
Hi Vimal,
Which version of HBase do you use? Can you also share your config file?
JM
2014-03-06 8:02 GMT-05:00 Vimal Jain vkj...@gmail.com:
Hi Jean,
I am not sure about this.
Whats the default policy ?
On Thu, Mar 6, 2014 at 6:30 PM, Jean-Marc Spaggiari
jean-m...@spaggiari.org
Hi,
I am using 0.94.17 and my config file is
property
namehbase.rootdir/name
valuehdfs://10.14.24.19:9000/hbase/value
/property
property
namehbase.cluster.distributed/name
Thanks Jean.
On Thu, Mar 6, 2014 at 6:41 PM, Jean-Marc Spaggiari jean-m...@spaggiari.org
wrote:
Default split policy for 0.94.17 is
IncreasingToUpperBoundRegionSplitPolicy. Since you did not set a specific
policy in your config file, that's the one used.
Comments from the class:
* Split
Message-
From: Nicolas Spiegelberg [mailto:nspiegelb...@fb.com]
Sent: Tuesday, November 22, 2011 10:59 PM
To: user@hbase.apache.org
Subject: Re: Region Splits
No. The purpose of major compactions is to merge dedupe
within a
region
boundary. Compactions will not alter region boundaries
: Region Splits
Mark,
Yes, your understanding is correct. If your keys are sequential (timestamps
etc), you will always be writing to the end of the table and older
regions will not get any writes. This is one of the arguments against using
sequential keys.
-ak
On Sun, Nov 20, 2011 at 11:33 AM, Mark
: Monday, November 21, 2011 7:25 AM
To: user@hbase.apache.org
Subject: Re: Region Splits
Mark,
Yes, your understanding is correct. If your keys are sequential
(timestamps
etc), you will always be writing to the end of the table and older
regions will not get any writes. This is one of the arguments
grows to 2GB size?
Thanks,
Srikanth
-Original Message-
From: Nicolas Spiegelberg [mailto:nspiegelb...@fb.com]
Sent: Tuesday, November 22, 2011 10:59 PM
To: user@hbase.apache.org
Subject: Re: Region Splits
No. The purpose of major compactions is to merge dedupe within a region
boundary
[mailto:nspiegelb...@fb.com]
Sent: Tuesday, November 22, 2011 10:59 PM
To: user@hbase.apache.org
Subject: Re: Region Splits
No. The purpose of major compactions is to merge dedupe within a region
boundary. Compactions will not alter region boundaries, except in the
case of splits where a compaction
regions grows to
2GB size?
Thanks,
Srikanth
-Original Message-
From: Nicolas Spiegelberg [mailto:nspiegelb...@fb.com]
Sent: Tuesday, November 22, 2011 10:59 PM
To: user@hbase.apache.org
Subject: Re: Region Splits
No. The purpose of major compactions is to merge dedupe within a region
space?
Is there a way to reorganize the regions so that each regions grows to
2GB size?
Thanks,
Srikanth
-Original Message-
From: Nicolas Spiegelberg [mailto:nspiegelb...@fb.com]
Sent: Tuesday, November 22, 2011 10:59 PM
To: user@hbase.apache.org
Subject: Re: Region Splits
-
From: Nicolas Spiegelberg [mailto:nspiegelb...@fb.com]
Sent: Tuesday, November 22, 2011 10:59 PM
To: user@hbase.apache.org
Subject: Re: Region Splits
No. The purpose of major compactions is to merge dedupe within a
region
boundary. Compactions will not alter region boundaries, except
]
Sent: Tuesday, November 22, 2011 10:59 PM
To: user@hbase.apache.org
Subject: Re: Region Splits
No. The purpose of major compactions is to merge dedupe within a
region
boundary. Compactions will not alter region boundaries, except in the
case of splits where a compaction is necessary to filter
Damn, I was hoping my understanding was flawed.
In your example I am guessing the addition of old_key suffix is to
prevent against any possible collision. Is that correct?
On 11/20/11 9:39 PM, Nicolas Spiegelberg wrote:
Sequential writes are also an argument for pre-splitting and using hash
Mark: you are correct about the old_key suffix. I'm assuming that you're
worried about this because of keyspace size, correct? The default
algorithm for pre-splitting assumes a 32-bit (4 byte) hash prefix, which
should be perfectly scalable for all use cases in the near future of
computing.
Hi there-
The last part of 6.3.2.3 is important: Expect tradeoffs
when designing rowkeys. Some of this stuff you just have to
prototype.
In terms of performance...
http://hbase.apache.org/book.html#keyvalue
.. if you have huge keys, you'll feel it there.
For the MR part, see...
Mark,
Yes, your understanding is correct. If your keys are sequential (timestamps
etc), you will always be writing to the end of the table and older
regions will not get any writes. This is one of the arguments against using
sequential keys.
-ak
On Sun, Nov 20, 2011 at 11:33 AM, Mark
Sequential writes are also an argument for pre-splitting and using hash
prefixing. In other words, presplit your table into N regions instead of
the default of 1 transform your keys into:
new_key = md5(old_key) + old_key
Using this method your sequential writes under the old_key are now spread
Have you run hbck to check all the regions’ assignment?
Maybe it will give you some helpful info.
Jieshan Bean
发件人: Sujee Maniyam [mailto:su...@sujee.net]
发送时间: 2011年6月15日 15:54
收件人: hbase-user
主题: region splits, but not assigned to other region servers
Hi all
I am loading data into a table
Hey Vidhya,
Anything interesting in the logs on the master?
It's possible that the master is slowly assigning out all the daughter
regions, and it's just taking a really long time since you loaded so many.
-Todd
On Thu, Aug 26, 2010 at 11:59 AM, Vidhyashankar Venkataraman
It's possible that the master is slowly assigning out all the daughter
regions, and it's just taking a really long time since you loaded so many.
I forgot to add it the last time.. But I think that's what is happening..
Master getting choked up because of a flash crowd of splits..
(Anyways, I
22 matches
Mail list logo
