[
https://issues.apache.org/jira/browse/HBASE-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12875476#action_12875476
]
HBase Review Board commented on HBASE-2001:
-------------------------------------------
Message from: "Andrew Purtell" <[email protected]>
bq. On 2010-06-02 23:28:50, stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/coprocessor/Coprocessor.java, line
140
bq. > <http://review.hbase.org/r/96/diff/4/?file=892#file892line140>
bq. >
bq. > Which table is this?
Any table. The idea is the coprocessor can create any private tables it needs
to implement its functionality.
bq. On 2010-06-02 23:28:50, stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/coprocessor/Coprocessor.java, line
27
bq. > <http://review.hbase.org/r/96/diff/4/?file=892#file892line27>
bq. >
bq. > What about unloading? You remember that conversation up on irc of
how loading is one thing but unloading w/o breakage is hard prob.
I'm not trying to tackle unloading yet.
However, classes are strongly bound to their classloader. We do instantiate a
classloader each time to load a coprocessor and we don't hold a reference to
the classloader. It is my understanding that when there are no more references
to the classes (no live objects), they and the classloader will be garbage
collected, though the JVM spec does not guarantee this the Sun JVM will do
this. Creating a new classloader and asking for the class again, presumably
from an updated jar, should load the new class -- a unit test can verify.
To help insure old classes don't leak via live objects hanging around, we could
consider a cooperative lifecycle management scheme like that used by OSGi:
http://en.wikipedia.org/wiki/OSGi.
bq. On 2010-06-02 23:28:50, stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/coprocessor/Coprocessor.java, line
150
bq. > <http://review.hbase.org/r/96/diff/4/?file=892#file892line150>
bq. >
bq. > Needs to be Writable?
No.
bq. On 2010-06-02 23:28:50, stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java,
line 29
bq. > <http://review.hbase.org/r/96/diff/4/?file=893#file893line29>
bq. >
bq. > This is a mixin you'd use if you want to be notified about
compactions, etc.?
This is for translating values found in a flush file into new values in the new
storefile being built by the compaction, or for dropping values.
bq. On 2010-06-02 23:28:50, stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/regionserver/CoprocessorHost.java,
line 85
bq. > <http://review.hbase.org/r/96/diff/4/?file=894#file894line85>
bq. >
bq. > What would this be used for? For CP to call out elsewhere on a
table?
The idea is the coprocessor can create any private tables it needs to implement
its functionality. But we want to mediate that, add access control, clean up
references when/if the cp is terminated (and perhaps unloaded).
bq. On 2010-06-02 23:28:50, stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/regionserver/CoprocessorHost.java,
line 237
bq. > <http://review.hbase.org/r/96/diff/4/?file=894#file894line237>
bq. >
bq. > Is this needed?
Why not.
bq. On 2010-06-02 23:28:50, stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java, line 245
bq. > <http://review.hbase.org/r/96/diff/4/?file=895#file895line245>
bq. >
bq. > Its always on then?
Yes, otherwise I have to wrap all calls to the cp host in HRegion with "if
(coprocessorHost != null) then", including the inner loops of the major and
minor compactors.
bq. On 2010-06-02 23:28:50, stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java, line
2885
bq. > <http://review.hbase.org/r/96/diff/4/?file=895#file895line2885>
bq. >
bq. > Who would want to get at this?
Tests. So probably this does not have to be public.
- Andrew
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/96/#review123
-----------------------------------------------------------
> Coprocessors: Colocate user code with regions
> ---------------------------------------------
>
> Key: HBASE-2001
> URL: https://issues.apache.org/jira/browse/HBASE-2001
> Project: HBase
> Issue Type: Sub-task
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Attachments: asm-3.2-bin.zip, asm-transformations.pdf,
> HBASE-2001-RegionObserver.patch, HBASE-2001.patch.gz
>
>
> Support user code that runs run next to each region in table. As regions
> split and move, coprocessor code should automatically move also.
> Use classloader which looks on HDFS.
> Associate a list of classes to load with each table. Put this in HRI so it
> inherits from table but can be changed on a per region basis (so then those
> region specific changes can inherited by daughters).
> Not completely arbitrary code, should require implementation of an interface
> with callbacks for:
> * Open
> * Close
> * Split
> * Compact
> * (Multi)get and scanner next()
> * (Multi)put
> * (Multi)delete
> Add method to HRegionInterface for invoking coprocessor methods and
> retrieving results.
> Add methods in o.a.h.h.regionserver or subpackage which implement convenience
> functions for coprocessor methods and consistent/controlled access to
> internals: store access, threading, persistent and ephemeral state, scratch
> storage, etc.
> GitHub: http://github.com/apurtell/hbase-coprocessor
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
