----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/876/#review1366 -----------------------------------------------------------
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java <http://review.cloudera.org/r/876/#comment4522> I would actually be -1 on having this call the get coprocessor hooks. Expected behavior (for me) would be that coprocessors hook into the client-side calls, not internal calls. For checkAndPut, are those Gets also wrapped by coprocessors? I suppose you could make an argument either way, but I'd err on the side of coprocessors attaching on client operations not internal ones. - Jonathan On 2010-09-19 18:35:17, Mingjie Lai wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > http://review.cloudera.org/r/876/ > ----------------------------------------------------------- > > (Updated 2010-09-19 18:35:17) > > > Review request for hbase. > > > Summary > ------- > > > The diff actually contains 2 seperate patches: HBase-2001 and the one for > (HBASE-2002+HBASE-2321). The reason is that HBase-2001's CommandTarget relies > on HBASE-2002 + HBASE-2321 which patches are still under review. I have to > include Gary's HBASE-2002, HBASE-2321 with this diff, since reviewboard is so > powerful :) and it disallow my diff to be based on some unchecked in patch. > > Both HBase-2001 and the dynamic RPC stuff are quite big patches. Total number > of lines are more than 7k. I turned back and forth, but still don't have a > good idea to create the patch in order to reduce the review pain. However > right now I'm putting the whole patch for all the 3 issues. Here the list of > file which are only related to coprocessor: > > src/main/java/org/apache/hadoop/hbase/coprocessor/BaseCommandTarget.java > src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java > src/main/java/org/apache/hadoop/hbase/coprocessor/Coprocessor.java > src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorEnvironment.java > src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorException.java > src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java > src/main/java/org/apache/hadoop/hbase/coprocessor/package-info.java > src/main/java/org/apache/hadoop/hbase/regionserver/CoprocessorHost.java > src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java > src/main/resources/hbase-default.xml > src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassloading.java > src/test/java/org/apache/hadoop/hbase/coprocessor/TestCommandTarget.java > src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java > src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java > src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java > > > ========================== > > (Here is a brief description. Please find much more details at the > package-info.java in the diff. I also post the package-info.html to > https://issues.apache.org/jira/browse/HBASE-2001 as an attachment.) > > > Coprocessors are code that runs in-process on each region server. Regions > contain references to the coprocessor implementation classes associated with > them. Coprocessor classes will be loaded either from local jars on the region > server's classpath or via the HDFS classloader. > > Multiple types of coprocessors are provided to provide sufficient flexibility > for potential use cases. Right now there are: > > * Coprocessor: provides region lifecycle management hooks, e.g., region > open/close/split/flush/compact operations. > * RegionObserver: provides hook for monitor table operations from client > side, such as table get/put/scan/delete, etc. > * CommandTarget: provides on demand triggers for any arbitrary function > executed at a region. One use case is column aggregation at region server. > > Coprocessor: > A coprocessor is required to implement Coprocessor interface so that > coprocessor framework can manage it internally. > > Another design goal of this interface is to provide simple features for > making coprocessors useful, while exposing no more internal state or control > actions of the region server than necessary and not exposing them directly. > > RegionObserver > If the coprocessor implements the RegionObserver interface it can observe and > mediate client actions on the region. > > CommandTarget: > Coprocessor and RegionObserver provide certain hooks for injecting user code > running at each region. These code will be triggerd with existing HTable and > HBaseAdmin operations at the certain hook points. > > Through CommandTarget and dynamic RPC protocol, you can define your own > interface communicated between client and region server, i.e., you can > specify new passed parameters and return types for a method. And the new > CommandTarget methods can be triggered by calling client side dynamic RPC > functions -- HTable.exec(...). > > Coprocess loading > A customized coprocessor can be loaded by two different ways, by > configuration, or by HTableDescriptor for a newly created table. > > (Currently we don't really have an on demand coprocessor loading machanism > for opened regions. However it should be easy to create a dedicated > CommandTarget for coprocessor loading) > > > This addresses bug HBase-2001. > http://issues.apache.org/jira/browse/HBase-2001 > > > Diffs > ----- > > src/main/java/org/apache/hadoop/hbase/client/Action.java 556ea81 > src/main/java/org/apache/hadoop/hbase/client/Exec.java PRE-CREATION > src/main/java/org/apache/hadoop/hbase/client/ExecResult.java PRE-CREATION > src/main/java/org/apache/hadoop/hbase/client/HConnection.java 65f7618 > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java > fbdec0b > src/main/java/org/apache/hadoop/hbase/client/HTable.java 0dbf263 > src/main/java/org/apache/hadoop/hbase/client/MultiAction.java c6ea838 > src/main/java/org/apache/hadoop/hbase/client/MultiResponse.java 91bd04b > src/main/java/org/apache/hadoop/hbase/client/RowRange.java PRE-CREATION > src/main/java/org/apache/hadoop/hbase/client/Scan.java 29b3cb0 > src/main/java/org/apache/hadoop/hbase/coprocessor/BaseCommandTarget.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/coprocessor/Coprocessor.java > PRE-CREATION > > src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorEnvironment.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorException.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/coprocessor/package-info.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 83f623d > src/main/java/org/apache/hadoop/hbase/ipc/ConnectionHeader.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/ipc/CoprocessorProtocol.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java PRE-CREATION > src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java 2b5eeb6 > src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java e23a629 > src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java e4c356d > src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java a4810a6 > src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java PRE-CREATION > src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java PRE-CREATION > src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java PRE-CREATION > src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/master/HMaster.java bccdc0e > src/main/java/org/apache/hadoop/hbase/regionserver/CoprocessorHost.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java fdef130 > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > f2e4e7c > src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java > bb3b382 > src/main/resources/hbase-default.xml 5452fd1 > src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassloading.java > PRE-CREATION > src/test/java/org/apache/hadoop/hbase/coprocessor/TestCommandTarget.java > PRE-CREATION > > src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java > PRE-CREATION > > src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java > PRE-CREATION > > src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java > PRE-CREATION > > src/test/java/org/apache/hadoop/hbase/regionserver/TestServerCustomProtocol.java > PRE-CREATION > > Diff: http://review.cloudera.org/r/876/diff > > > Testing > ------- > > > Thanks, > > Mingjie > >
