[
https://issues.apache.org/jira/browse/HADOOP-5640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698303#action_12698303
]
Todd Lipcon commented on HADOOP-5640:
-------------------------------------
Hacked this up today, but I just realized there may be a big downside to the
multithread approach. While it's nice to decouple the execution of the plugin
with the execution of the service, it's likely to introduce a lot of hairy
synchronization bugs. As a specific example, the Thrift datanode plugin
accesses the datanode.dnRegistration member without synchronization,and the
datanode potentially modifies this instance when re-registering with the
namenode after a disconnection. My bet is that there other more subtle
instances where this is the case.
Given this, I think I'd prefer to say "plugin authors should take care that
they do not block or create potential deadlock situations with their host
services. Install plugins at your own risk" rather than force them to execute
in separate threads everywhere.
The other option is to look over all public members (and those available
through accessors) to be sure that anything allowed to escape out of the daemon
is an immutable snapshot of the internal state.
> Allow ServicePlugins to hook callbacks into key service events
> --------------------------------------------------------------
>
> Key: HADOOP-5640
> URL: https://issues.apache.org/jira/browse/HADOOP-5640
> Project: Hadoop Core
> Issue Type: Improvement
> Components: util
> Reporter: Todd Lipcon
>
> HADOOP-5257 added the ability for NameNode and DataNode to start and stop
> ServicePlugin implementations at NN/DN start/stop. However, this is
> insufficient integration for some common use cases.
> We should add some functionality for Plugins to subscribe to events generated
> by the service they're plugging into. Some potential hook points are:
> NameNode:
> - new datanode registered
> - datanode has died
> - exception caught
> - etc?
> DataNode:
> - startup
> - initial registration with NN complete (this is important for HADOOP-4707
> to sync up datanode.dnRegistration.name with the NN-side registration)
> - namenode reconnect
> - some block transfer hooks?
> - exception caught
> I see two potential routes for implementation:
> 1) We make an enum for the types of hookpoints and have a general function in
> the ServicePlugin interface. Something like:
> {code:java}
> enum HookPoint {
> DN_STARTUP,
> DN_RECEIVED_NEW_BLOCK,
> DN_CAUGHT_EXCEPTION,
> ...
> }
> void runHook(HookPoint hp, Object value);
> {code}
> 2) We make classes specific to each "pluggable" as was originally suggested
> in HADDOP-5257. Something like:
> {code:java}
> class DataNodePlugin {
> void datanodeStarted() {}
> void receivedNewBlock(block info, etc) {}
> void caughtException(Exception e) {}
> ...
> }
> {code}
> I personally prefer option (2) since we can ensure plugin API compatibility
> at compile-time, and we avoid an ugly switch statement in a runHook()
> function.
> Interested to hear what people's thoughts are here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
