How common is Hbase throttling?

2017-11-14 Thread Sumit Nigam
Hi, Hbase allows for throttling at any of the following levels (or a combination): namespace, table, user. Further, the throttling can be on number of requests and/or volume of data.  So, I’d like to understand:  1) Is throttling enabled by most of the end users?  2) If yes, which of the above

Re: Multi-homed hosts with Hbase

2016-11-30 Thread Sumit Nigam
Hi, Any insights into this issue? Thanks,Sumit From: Sumit Nigam <sumit_o...@yahoo.com.INVALID> To: Hbase-user <user@hbase.apache.org> Sent: Tuesday, November 29, 2016 10:02 PM Subject: Multi-homed hosts with Hbase Hi, I am using multi-homed kerberized hosts wit

Multi-homed hosts with Hbase

2016-11-29 Thread Sumit Nigam
Hi, I am using multi-homed kerberized hosts with hbase 0.98. The /etc/hosts files are uniformly containing FQDNs for cluster nodes. However, region server fails to register with master server with the error: 2016-11-17 15:05:47,700 WARN  [regionserver54723] regionserver.HRegionServer: error

Re: Hbase throttling issues

2016-09-25 Thread Sumit Nigam
in 1.2, but not sure why this quota exceeded error with READ_SIZE then shows up. Thanks,Sumit From: Guanghao Zhang <zghao...@gmail.com> To: user@hbase.apache.org; Sumit Nigam <sumit_o...@yahoo.com> Sent: Thursday, September 22, 2016 12:49 PM Subject: Re: Hbase throttling issues

Re: Hbase throttling issues

2016-09-21 Thread Sumit Nigam
No, did not try request number. I want to use size as my throttling factor.  Thanks again! From: Guanghao Zhang <zghao...@gmail.com> To: user@hbase.apache.org; Sumit Nigam <sumit_o...@yahoo.com> Sent: Thursday, September 22, 2016 9:28 AM Subject: Re: Hbase throttling issues

Re: Hbase throttling issues

2016-09-21 Thread Sumit Nigam
: Guanghao Zhang <zghao...@gmail.com> To: user@hbase.apache.org; Sumit Nigam <sumit_o...@yahoo.com> Sent: Thursday, September 22, 2016 7:37 AM Subject: Re: Hbase throttling issues All scan operations were throttled? Now it use the avg size of all previous operations to check

Hbase throttling issues

2016-09-21 Thread Sumit Nigam
Hi Hbase experts, Our application is unable to scan or read from hbase tables, when throttling is set. We are getting ThrottlingException every time. The error is seen more frequently when the number of hbase pre splits is increased. The size tables for which this error is showing is empty (

Re: Region server crashes after GC pause time

2016-04-22 Thread Sumit Nigam
Hi Karthik, As Ding said, eventually you need to watch your heap objects to understand what produces so many objects in the first place. CMS occupancy % defaults to 70%. Default values of block cache and in-memory memstore sizes add up to around 75-80%. Which means you could get CMS to start

Re: Major compaction

2016-04-04 Thread Sumit Nigam
t;j...@merkleinc.com> To: "user@hbase.apache.org" <user@hbase.apache.org> Cc: Sumit Nigam <sumit_o...@yahoo.com> Sent: Monday, April 4, 2016 11:07 PM Subject: RE: Major compaction I wrote a small program to do MC in a "smart" way here: https://github.com/

Major compaction

2016-04-04 Thread Sumit Nigam
Hi, Are there major overheads to running major compaction frequently? As much as I know, it produces one Hfile for a region and processes delete markers and version related drops. So, if this process has happened once say. a few mins back then another major compaction should ideally not cause

Re: Compacting same table

2016-04-03 Thread Sumit Nigam
iven Hfile, as many KVs would be read at one time? So, if I increase memstore size, I should also increase this value because default value I believe is 10, which is probably too less. Thanks,Sumit From: Ted Yu <yuzhih...@gmail.com> To: "user@hbase.apache.org" <user@h

Compacting same table

2016-04-03 Thread Sumit Nigam
Hi, I have been informed that compacting (manual) the same hbase table takes same amount of time even when done in quick succession. This seems counter-intuitive because an already compacted table should not take same amount of time. Also, what is the use of hbase.hstore.compaction.kv.max

Re: Table copies in hbase

2016-01-20 Thread Sumit Nigam
n of hbase you are using? Regarding you question what will > > happen in case operation fails in case of cloning snapshot i have never > > experience such event if snapshot was not corrupted. In case of CopyTable > > mr i assume that data will be partially copied. In any case your s

Table copies in hbase

2016-01-19 Thread Sumit Nigam
Hi, I have a a few tables in hbase which I would like to create copies of. The tables will be in same cluster. I wanted to know based on experience, which one of the options is better: 1. copyTable2. snapshot3. Any other? Besides efficiency aspect, I am also interested in knowing what happens

hbase.client.retries.number

2015-11-26 Thread Sumit Nigam
Hi, Need some help/ inputs. I have set hbase.client.retries.number as 40. Double checked this to make sure it is 40 and not 400 as reported below! However, I notice following in my logs: 2015-11-19 16:35:02,687 WARN  [htable-pool5-t1] client.AsyncProcess: #3, table=ldmns:indx_parameterstore,

Re: About exceptions

2015-11-20 Thread Sumit Nigam
.compute.internal,40359,1447816123180, tracking started Tue Nov 17 22:26:27 PST 2015, retrying after 10046 ms, replay 2 ops. Thanks,Sumit From: Ted Yu <yuzhih...@gmail.com> To: Sumit Nigam <sumit_o...@yahoo.com> Cc: "user@hbase.apache.org" <user@hbase.apache.org>

Re: About exceptions

2015-11-15 Thread Sumit Nigam
be related to Hbase code? And any advise on if I can somehow avoid it in first place?  Thanks,Sumit From: Ted Yu <yuzhih...@gmail.com> To: Sumit Nigam <sumit_o...@yahoo.com> Cc: "user@hbase.apache.org" <user@hbase.apache.org> Sent: Sunday, November 15, 2015

Re: About exceptions

2015-11-15 Thread Sumit Nigam
) From: Ted Yu <yuzhih...@gmail.com> To: "user@hbase.apache.org" <user@hbase.apache.org>; Sumit Nigam <sumit_o...@yahoo.com> Sent: Sunday, November 15, 2015 10:50 AM Subject: Re: About exceptions bq. TableNotEnabledExceptionTableNotFoundExceptionIOExcepti

About exceptions

2015-11-14 Thread Sumit Nigam
Hi, There are some exceptions which I face intermittently with Hbase and I thought some help from experts online can really help me. These are: TableNotEnabledExceptionTableNotFoundExceptionIOException - TableNamespaceManager isn't ready to serve One of the reasons I can see for this seems to

Hbck and exceptions

2015-09-23 Thread Sumit Nigam
Hi, I have a few questions/ observations with respect to some of the Hbase exceptions.  I have used various hbck options with varied success in case of Hbase master failing to startup after a shutdown. However, it has been more of trial and error. Is there any recommendation document as to what