I have post images to hosting service and share again!! --- Hi. This is Ui Jin, Lim in Korea, LG CNS
We had setup and are operating hbase 0.98.13 on our customer, iptv Company. Our customer claim about this spike. Following is our status. . We had been operating with BlockCache(heap) for 3 months. The ratio of above 100ms has been reached 6.69% of total transaction. The following graph show the trend of executions, above 100ms per minute, at June 17th. (This is the response time of 2~3 Hbase transactions) <Click to see Graph> http://i57.tinypic.com/mwcvtj.jpg - After Hbase restarting at June 18th, The spikes had been reduced dramatically. But its ratio had been increasing continuously. <Click to see Graph> http://i57.tinypic.com/71pfee.jpg - We had adopted off-heap bucket cache at August 19th. But the increasing does not disappear. <Click to see Graph> http://i61.tinypic.com/2aezazt.jpg http://i58.tinypic.com/svm32x.jpg - We set HBase to log slow query which have above 10ms execution time. All RegionServer logs show increasing its counts. <Click to see Graph> http://i59.tinypic.com/68u1ci.jpg What we concern is not the spike itself but its incremental trend. There is no increment of transactions and dataset. I ask your thoughts about this issue. -----Original Message----- From: Vladimir Rodionov [mailto:[email protected]] Sent: Thursday, October 08, 2015 9:50 AM To: [email protected] Subject: Re: Asking about the trend of increasing latency, hbase spikes. That is definitely, interesting graphs, but we do not see them. Can you host them somewhere and post here the links? -Vlad On Wed, Oct 7, 2015 at 5:38 PM, Biju N <[email protected]> wrote: > Looks like the attachments are not viewable. > > On Wed, Oct 7, 2015 at 8:35 PM, UI-JIN LIM <[email protected]> wrote: > > > Hi. This is Ui Jin, Lim in Korea, LG CNS > > > > > > > > We had setup and are operating hbase 0.98.13 on our customer, iptv > > Company. > > > > Our customer claim about this spike. Following is our status. > > > > > > > > . We had been operating with BlockCache(heap) for 3 months. The > > ratio of above 100ms has been reached 6.69% of total transaction. > > > > The following graph show the trend of executions, above 100ms per > > minute, at June 17th. (This is the response time of 2~3 Hbase > > transactions) > > > > [image: cid:[email protected]] > > > > > > > > - After Hbase restarting at June 18th, The spikes had been reduced > > dramatically. But its ratio had been increasing continuously. > > > > [image: cid:[email protected]] > > > > > > > > - We had adopted off-heap bucket cache at August 19th. But the > > increasing does not disappear. > > > > [image: cid:[email protected]] > > > > [image: cid:[email protected]] > > > > > > > > - We set HBase to log slow query which have above 10ms execution time. > All > > RegionServer logs show increasing its counts. > > > > > > > > [image: cid:[email protected]] > > > > > > > > > > > > What we concern is not the spike itself but its incremental trend. > > > > There is no increment of transactions and dataset. > > > > I ask your thoughts about this issue. > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------- > > > > *UI JIN, LIM* > > > > *DNA 3.0** - **LG CNS* > > > > Data Architecture Advisory > > > > Big Data Technical Support Team > > > > Mobile : +82) 010-8307-1210 > > > > > > >
