I think we should try to make AggregateProtocol work first. I am not with computer at this moment. Will look at the code when I get back.
The table below isn't too big. Experimenting with a smaller table would be nice. Cheers On Jan 1, 2012, at 11:26 AM, Royston Sellman <[email protected]> wrote: > Hi Ted, > > I think 0 is the only value we ever see (I'll check tomorrow: the server is > down right now). Our table has 557,000 rows. I'll try a much shorter table > tomorrow. > > Yes, we have RS running on the NN, but it's a test cluster and we are used to > it :) > > Do you think using AggregationProtocol is the best strategy for the case > where we want to use basic SQL-style functions like SUM, AVG, STD, MIN, MAX? > Do you think there is a better strategy? > > Many thanks, > Royston > > > On 1 Jan 2012, at 17:58, Ted Yu wrote: > >> Royston: >> Happy New Year to you too. >> >>>> java.net.SocketTimeoutException: Call to namenode/10.0.0.235:60020failed on >> >> It seems the namenode above actually refers to a region server. This is a >> little bit confusing :-) >> >> The sum value below is 0. >> Have you ever seen a value greater than 0 ? >> >> How many rows are there in this CF:CQ ? >> The timeout was reported earlier by other people where there're many rows >> in the table. >> >> There is a JIRA to provide streaming support for coprocessor but the >> development there has stalled. >> >> Cheers >> >> On Sun, Jan 1, 2012 at 9:35 AM, Royston Sellman < >> [email protected]> wrote: >> >>> Hi Gary and Ted, >>> >>> Royston (Tom's colleague) here. Back onto this after the Christmas/New Year >>> break. >>> >>> Many thanks for your help so far. We enabled our database via your >>> hbase-site.xml mod and were able to move on. to other errors. But I think >>> we >>> are now actually getting an aggregation partially calculated on our table >>> (this feels like progress). The details: >>> >>> On running our client we now get this exception: >>> 11/12/31 17:51:09 WARN >>> client.HConnectionManager$HConnectionImplementation: Error executing for >>> row >>> >>> java.util.concurrent.ExecutionException: >>> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after >>> attempts=10, exceptions: >>> Sat Dec 31 17:41:30 GMT 2011, >>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1fc4f0f8, >>> java.net.SocketTimeoutException: Call to namenode/10.0.0.235:60020 failed >>> on >>> socket timeout exception: java.net.SocketTimeoutException: 60000 millis >>> timeout while waiting for channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/10.0.0.235:59999 >>> remote=namenode/10.0.0.235:60020] >>> (8 more of these, making for 10 tries) >>> Sat Dec 31 17:51:09 GMT 2011, >>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1fc4f0f8, >>> java.net.SocketTimeoutException: Call to namenode/10.0.0.235:60020 failed >>> on >>> socket timeout exception: java.net.SocketTimeoutException: 60000 millis >>> timeout while waiting for channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/10.0.0.235:59364 >>> remote=namenode/10.0.0.235:60020] >>> >>> at >>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) >>> at java.util.concurrent.FutureTask.get(FutureTask.java:83) >>> at >>> >>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation. >>> processExecs(HConnectionManager.java:1465) >>> at >>> org.apache.hadoop.hbase.client.HTable.coprocessorExec(HTable.java:1555) >>> at >>> >>> org.apache.hadoop.hbase.client.coprocessor.AggregationClient.sum(Aggregation >>> Client.java:229) >>> at EDRPAggregator.testSumWithValidRange(EDRPAggregator.java:51) >>> at EDRPAggregator.main(EDRPAggregator.java:77) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39 >>> ) >>> at >>> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl >>> .java:25) >>> at java.lang.reflect.Method.invoke(Method.java:597) >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >>> >>> >>> Looking at the log (.regionserver-namenode.log) I see this debug message: >>> >>> 2011-12-31 17:42:23,472 DEBUG >>> org.apache.hadoop.hbase.coprocessor.AggregateImplementation: Sum from this >>> region is EDRPTestTbl,,1324485124322.7b9ee0d113db9b24ea9fdde90702d006.: 0 >>> >>> Where the sum value looks reasonable which makes me think the sum of a >>> CF:CQ >>> worked. But I never see this value on stdout. Then I see this warning: >>> >>> 2011-12-31 17:42:23,476 WARN org.apache.hadoop.ipc.HBaseServer: >>> (responseTooSlow): {"processingtimems":113146,"call":"execCoprocess$ >>> 2011-12-31 17:42:23,511 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server >>> Responder, call execCoprocessor([B@4b22fad6, getSum(org.$ >>> 2011-12-31 17:42:23,515 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server >>> handler 1 on 60020 caught: java.nio.channels.ClosedChann$ >>> at >>> sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133) >>> at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324) >>> at >>> org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1651) >>> at >>> >>> org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServe >>> r.java:924) >>> at >>> >>> org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java >>> :1003) >>> at >>> >>> org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer >>> .java:409) >>> at >>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1345) >>> >>> Have we missed out some step in the HBase/RegionServerconfig? Or is our >>> client code still deficient? >>> >>> Can you offer any suggestions? Is there any example code for the new >>> Aggregations stuff. >>> >>> Thanks and Happy New Year to you guys, >>> >>> Royston (and Tom). >>> >>> (HBase 0.92, Hadoop 1.0) >>> >>> >>> -----Original Message----- >>> From: Gary Helmling [mailto:[email protected]] >>> Sent: 23 December 2011 18:06 >>> To: [email protected] >>> Subject: Re: AggregateProtocol Help >>> >>> Hi Tom, >>> >>> The test code is not really the best guide for configuration. >>> >>> To enable the AggregateProtocol on all of your tables, add this to the >>> hbase-site.xml for the servers in your cluster: >>> >>> <property> >>> <name>hbase.coprocessor.user.region.classes</name> >>> >>> <value>org.apache.hadoop.hbase.coprocessor.AggregateImplementation</value> >>> </property> >>> >>> If you only want to use the aggregate functions on a specific table (or >>> tables), then you can enable that individually for the table from the >>> shell: >>> >>> 1) disable the table >>> hbase> disable 'EDRP7' >>> >>> 2) add the coprocessor >>> hbase> alter 'EDRP7', METHOD => 'table_att', >>> >>> >>> 'coprocessor'=>'|org.apache.hadoop.hbase.coprocessor.AggregateImplementation >>> ||' >>> >>> (Note that the pipes in the value string are required) >>> >>> 3) re-enable the table >>> hbase> enable 'EDRP7' >>> >>> >>> Either way should work. With the second approach you will see the >>> coprocessor listed when you describe the table from the shell, as Ted >>> mentioned. With the first approach you will not, but it should be loaded >>> all the same. >>> >>> --gh >>> >>> >>> On Fri, Dec 23, 2011 at 7:04 AM, Ted Yu <[email protected]> wrote: >>>> I don't know why you chose HBaseTestingUtility to create the table. >>>> I guess you followed test code example. >>>> >>>> At least you should pass the conf to this ctor: >>>> public HBaseTestingUtility(Configuration conf) { >>>> >>>> If coprocessor was installed correctly, you should see something >>>> like(from >>>> HBASE-5070): >>>> coprocessor$1 => >>>> '|org.apache.hadoop.hbase.constraint.ConstraintProcessor|1073741823|' >>>> >>>> Cheers >>>> >>>> On Fri, Dec 23, 2011 at 3:02 AM, Tom Wilcox <[email protected]> >>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I am not sure how we load the AggregateImplementation into the table. >>>>> When we are creating a table, we use the same functions as the test as >>> follows... >>>>> >>>>> ... >>>>>> conf.set(CoprocessorHost.REGION_COPROCESSOR_CONF_KEY, >>>>>> >>>>>> "org.apache.hadoop.hbase.coprocessor.AggregateImplementation"); >>>>>> >>>>>> // Utility.CreateHBaseTable(conf, otherArgs[1], >>>>> otherArgs[2], >>>>>> true); >>>>>> >>>>>> HBaseTestingUtility util = new HBaseTestingUtility(); >>>>>> HTable table = util.createTable(EDRP_TABLE, >>>>>> EDRP_FAMILY); >>>>>> >>>>>> AggregationClient aClient = new >>>>>> AggregationClient(conf); >>>>> ... >>>>> >>>>> Running DESCRIBE on a table produced shows the following output: >>>>> >>>>> hbase(main):002:0> describe 'EDRP7' >>>>> DESCRIPTION >>>>> ENABLED >>>>> {NAME => 'EDRP7', FAMILIES => [{NAME => 'advanceKWh', BLOOMFILTER => >>>>> 'NONE', REPLICATION_SCOPE => '0', VERSIONS => true >>>>> '3', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => >>>>> '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', B >>>>> LOCKCACHE => 'true'}]} >>>>> >>>>> We are using the tip of 0.92 (cloned from the Git repo). See the >>>>> version string below: >>>>> >>>>> hbase(main):005:0> version >>>>> 0.92.0, r1208286, Thu Dec 15 13:16:03 GMT 2011 >>>>> >>>>> We would really appreciate an example of how to create a table that >>>>> is enabled to handle Aggregation). >>>>> >>>>> Thanks >>>>> >>>>> >>>>> ________________________________________ >>>>> From: Ted Yu [[email protected]] >>>>> Sent: 22 December 2011 17:03 >>>>> To: [email protected] >>>>> Subject: Re: AggregateProtocol Help >>>>> >>>>> Have you loaded AggregateImplementation into your table ? >>>>> Can you show us the contents of the following command in hbase shell: >>>>> describe 'your-table' >>>>> >>>>> BTW are you using the tip of 0.92 ? >>>>> HBASE-4946 would be of help for dynamically loaded coprocessors which >>>>> you might use in the future. >>>>> >>>>> Cheers >>>>> >>>>> On Thu, Dec 22, 2011 at 8:09 AM, Tom Wilcox <[email protected]> >>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> We are trying to use the aggregation functionality in HBase 0.92 >>>>>> and we have managed to get the test code working using the following >>> command: >>>>>> >>>>>> java -classpath junit-4.10.jar:build/*:$HBASELIBS/* >>>>>> org.junit.runner.JUnitCore >>>>>> org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol >>>>>> >>>>>> Closer inspection of this test class has revealed that it uses a >>>>>> mini DFS cluster to populate and run the tests. These tests return >>> successfully. >>>>>> >>>>>> However, when we attempt to run similar code on our development >>>>>> HDFS cluster we experience the following error: >>>>>> >>>>>> 11/12/22 15:46:28 WARN >>>>>> client.HConnectionManager$HConnectionImplementation: Error >>>>>> executing for >>>>> row >>>>>> java.util.concurrent.ExecutionException: >>>>>> org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolException: >>>>>> org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolException: No >>>>> matching >>>>>> handler for protocol >>>>> org.apache.hadoop.hbase.coprocessor.AggregateProtocol >>>>>> in region EDRPTestTbl,,1324485124322.7b9ee0d113db9b24ea9fdde90702d006. >>>>>> at >>>>>> org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:4010 >>>>>> ) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HR >>>>> egionServer.java:3040) >>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >>>>> Method) >>>>>> at >>>>>> >>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. >>>>> java:39) >>>>>> at >>>>>> >>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces >>>>> sorImpl.java:25) >>>>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpc >>>>> Engine.java:364) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java: >>>>> 1325) >>>>>> [sshexec] >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) >>>>>> at >>>>>> java.util.concurrent.FutureTask.get(FutureTask.java:83) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen >>>>> tation.processExecs(HConnectionManager.java:1465) >>>>>> at >>>>>> org.apache.hadoop.hbase.client.HTable.coprocessorExec(HTable.java:1 >>>>>> 555) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.client.coprocessor.AggregationClient.sum(Aggr >>>>> egationClient.java:229) >>>>>> at >>>>>> EDRPAggregator.testSumWithValidRange(EDRPAggregator.java:51) >>>>>> at EDRPAggregator.main(EDRPAggregator.java:77) >>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >>>>> Method) >>>>>> at >>>>>> >>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. >>>>> java:39) >>>>>> at >>>>>> >>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces >>>>> sorImpl.java:25) >>>>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >>>>>> Caused by: >>>>>> org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolException: >>>>>> org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolException: No >>>>> matching >>>>>> handler for protocol >>>>> org.apache.hadoop.hbase.coprocessor.AggregateProtocol >>>>>> in region EDRPTestTbl,,1324485124322.7b9ee0d113db9b24ea9fdde90702d006. >>>>>> at >>>>>> org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:4010 >>>>>> ) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HR >>>>> egionServer.java:3040) >>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >>>>> Method) >>>>>> at >>>>>> >>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. >>>>> java:39) >>>>>> at >>>>>> >>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces >>>>> sorImpl.java:25) >>>>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpc >>>>> Engine.java:364) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java: >>>>> 1325) >>>>>> [sshexec] >>>>>> at >>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>>>>> Method) >>>>>> at >>>>>> >>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct >>>>> orAccessorImpl.java:39) >>>>>> at >>>>>> >>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC >>>>> onstructorAccessorImpl.java:27) >>>>>> at >>>>>> java.lang.reflect.Constructor.newInstance(Constructor.java:513) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException( >>>>> RemoteExceptionHandler.java:96) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen >>>>> tation.translateException(HConnectionManager.java:1651) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen >>>>> tation.getRegionServerWithRetries(HConnectionManager.java:1327) >>>>>> at >>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker.invoke(ExecRPCInvoker.ja >>>>>> va:79) >>>>>> at $Proxy3.getSum(Unknown Source) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.client.coprocessor.AggregationClient$4.call(A >>>>> ggregationClient.java:233) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.client.coprocessor.AggregationClient$4.call(A >>>>> ggregationClient.java:230) >>>>>> at >>>>>> >>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen >>>>> tation$4.call(HConnectionManager.java:1453) >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>>>>> at >>>>>> java.util.co
