puts will block at a point if the servers cannot follow. See the settings starting with hbase.client in the hbase book. As a first try, I would recommend to set hbase.client.max.perserver. tasks to 1 in the client. You may also want to change the buffer size ( hbase.client.write.buffer)...
On Wed, Jun 18, 2014 at 12:58 PM, Li Li <[email protected]> wrote: > and also there so many Puts maintained by background hbase threads > that consuming too much resources > > On Wed, Jun 18, 2014 at 6:54 PM, Li Li <[email protected]> wrote: > > I mean client slow itself down. e.g. > > my client code(one of many threads) > > while(true){ > > // process data and generate data need to insert to hbase > > List<Put> puts=...; > > table.put(puts); > > } > > > > the client (I mean threads created by hbase api, not by codes) is > > slowed down by retries. But if table.put(puts) don't block. my codes > > will send requests. And the server is too busy and gc stop the world > > and zookeeper don't receive heartbeat and regards it as dead > > > > On Wed, Jun 18, 2014 at 6:17 PM, Nicolas Liochon <[email protected]> > wrote: > >> What do you mean by down? Does it crash? > >> > >> The server does not block on 0.96, it immediately sends back an > exception. > >> (See HBASE-9467) > >> The client is implicitly slowed down by the retries, w/o blocking on the > >> server. It's managed by the hbase client itself, and it's transparent > for > >> the client application. You've got some logs (info level), to tell you > that > >> something is happening, but that's it. > >> > >> You can configure the client to be less more or less verbose > >> (hbase.client.start.log.errors.counter) or more or less pushy on the > server > >> (for example hbase.client.max.perserver.tasks). > >> > >> Cheers, > >> > >> Nicolas > >> > >> > >> On Wed, Jun 18, 2014 at 11:59 AM, Li Li <[email protected]> wrote: > >> > >>> hi all, > >>> the hbase client send too much requests and the some region server > >>> down. > >>> 1. region server down because of gc pause > >>> I can see it from log: > >>> [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM > >>> or host machine (eg GC): pause of approximately 3056ms > >>> I can adjust zookeeper config > >>> 2. as a client, how can I know server is busy? > >>> I am using hbase 0.96 with hadoop1. I can see log such as: > >>> org.apache.hadoop.hbase.client.AsyncProcess #178, table=vc2.url_db, > >>> attempt=13/35 SUCCEEDED on mphbase1,60020,1403084938641, tracking > >>> started Wed Jun 18 17:31:10 CST 2014 > >>> I googled and found > >>> > >>> > http://apache-hbase.679495.n3.nabble.com/RegionTooBusyException-Above-memstore-limit-td4056339.html > >>> it says in hbase 0.94, if the server is busy, it will block > >>> the client request. That's what I want, if the client send too much > >>> request than server can handle, it will be blocked. But in 0.96, it > >>> seems not. > >>> I did not find any exception in client. So I can't slow the > clien > >>> down. > >>> >
