>From HBase 0.98, there have been changes going into the Rest gateway, mainly more scan support.
There seems to be a change in the way the url table/rowkey* is executed on the Rest gateway. in pre-0.96, we set the startKey = rowkey and endKey = rowkey + one byte of 255 on Rest gateway in the Scan sent to the servers. In the newer versions, table/rowkey* seems to treated with a prefixFilter. This may explain why you are seeing slower response. Jerry On Thu, Dec 3, 2015 at 1:12 PM, Sutton, Ben <[email protected]> wrote: > We are currently experiencing an issue with HBase 1.1.1 through the REST > API whereby we are getting a very slow (> 2minutes) /non-existent response > when hitting the Rest API with a globbing option i.e. http:// > <HBase_Rest>:<HBase_Rest_Port/table/key*<http:// > %3cHBase_Rest%3e:%3cHBase_Rest_Port/table/key*> > > > > We are able to return each individual row for the key with no problem i.e. > http://<HBase_Rest>:<HBase_Rest_Port/table/key01<http://%3cHBase_Rest%3e:%3cHBase_Rest_Port/table/key01> > or > http://<HBase_Rest>:<HBase_Rest_Port/table/key02<http://%3cHBase_Rest%3e:%3cHBase_Rest_Port/table/key02> > etc. > > We have also found if we load in a third of the data to hbase we have no > trouble with the above. > > Facing this issue with 1.7billion row hbase table - we are able to > retrieve data for initial rows keys through to 1/10 of the region with a > slowly degrading performance after this point until no result. > > This is with 19 regionservers with each having at least 5 regions. > > Seeing the below error: > Failed after attempts=36, exceptions: > Tue Dec 01 12:51:29 GMT 2015, null, java.net.SocketTimeoutException: > callTimeout=60000, callDuration=60303: row '$ROW' on table '$TABLE' at > region=$TABLE,$ROW,1416253073158.2f25cef5708de354950561f4a399da36., > hostname=$LOCAL_HOST,60020,1448892673650, seqNum=444047093 > (through reference chain: > org.apache.hadoop.hbase.rest.CellSetModelStream["Row"]) > 2015-12-01 12:51:29,405 ERROR [1516838186@qtp-258535644-213] mortbay.log: > /$TABLE/KEY.* > java.lang.RuntimeException: > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after > attempts=36, exceptions: > Tue Dec 01 12:51:29 GMT 2015, null, java.net.SocketTimeoutException: > callTimeout=60000, callDuration=60303: row $ROW on table $TABLE at > region=$TABLE,04079290507.TiVo,1416253073158.2f25cef5708de354950561f4a399da36., > hostname=$LOCAL_HOST,60020,1448892673650, seqNum=444047093 > > at > org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:97) > at > org.apache.hadoop.hbase.rest.TableScanResource$1$1.hasNext(TableScanResource.java:82) > at > com.sun.xml.bind.v2.runtime.reflect.Lister$CollectionLister$1.hasNext(Lister.java:285) > at > com.sun.xml.bind.v2.runtime.property.ArrayElementProperty.serializeListBody(ArrayElementProperty.java:137) > at > com.sun.xml.bind.v2.runtime.property.ArrayERProperty.serializeBody(ArrayERProperty.java:159) > at > com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeBody(ClassBeanInfoImpl.java:344) > at > com.sun.xml.bind.v2.runtime.XMLSerializer.childAsSoleContent(XMLSerializer.java:597) > at > com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeRoot(ClassBeanInfoImpl.java:328) > at > com.sun.xml.bind.v2.runtime.XMLSerializer.childAsRoot(XMLSerializer.java:498) > at > com.sun.xml.bind.v2.runtime.MarshallerImpl.write(MarshallerImpl.java:320) > at > com.sun.xml.bind.v2.runtime.MarshallerImpl.marshal(MarshallerImpl.java:249) > at > javax.xml.bind.helpers.AbstractMarshallerImpl.marshal(AbstractMarshallerImpl.java:95) > at > com.sun.jersey.core.provider.jaxb.AbstractRootElementProvider.writeTo(AbstractRootElementProvider.java:179) > at > com.sun.jersey.core.provider.jaxb.AbstractRootElementProvider.writeTo(AbstractRootElementProvider.java:157) > at > com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:306) > at > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1437) > at > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349) > at > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339) > at > com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) > at > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537) > at > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221) > at > org.apache.hadoop.hbase.rest.filter.GzipFilter.doFilter(GzipFilter.java:76) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) > at > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) > at > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) > at > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767) > at > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) > at org.mortbay.jetty.Server.handle(Server.java:326) > at > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) > at > org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) > at > org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) > at > org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) > Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: > Failed after attempts=36, exceptions: > > Anyone have any ideas? > > Thanks, > > Ben > > > -------------------------------------------------------------------- > Save Paper - Do you really need to print this e-mail? > > Visit www.virginmedia.com for more information, and more fun. > > This email and any attachments are or may be confidential and legally > privileged > and are sent solely for the attention of the addressee(s). If you have > received this > email in error, please delete it from your system: its use, disclosure or > copying is > unauthorised. Statements and opinions expressed in this email may not > represent > those of Virgin Media. Any representations or commitments in this email are > subject to contract. > > Registered office: Media House, Bartley Wood Business Park, Hook, > Hampshire, RG27 9UP > Registered in England and Wales with number 2591237 >
