[
https://issues.apache.org/jira/browse/HADOOP-5257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683800#action_12683800
]
Carlos Valiente commented on HADOOP-5257:
-----------------------------------------
Thanks for your comments, Steve.
bq.1. Looking at the code, the plugin interface is very like the service
lifecycle in HADOOP-3628; start, stop, and such like. There's no reason why we
couldn't use the same lifecycle interface for plugins as the services themselves
Sounds very reasonable. Just one thing: For my current prototype of the Thrift
Namenode plugin, I use the reference to the
{{o.a.h.hdfs.server.namenode.NameNode}} instance passed as the parameter to the
{{ServicePlugin.start(server)}}. Something like this:
{code}
class NamenodePlugin implements ServicePlugin {
NameNode namenode;
void start(Object server) {
this.namenode = (NameNode)server;
}
// [..]
/* A sample method of the Thrift interface */
public void enterSafeMode() {
namenode.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
}
{code}
How could that be achieved in the context of HADOOP-3628?
All in all I'm happy to base this patch on HADOOP-3628. Are you aiming for 0.21
with your stuff, Steve? If HADOOP-3628 were to be included in 0.21, that would
make things very easy for my Thrift stuff.
bq.2. when you aggregate, you soon reach the policy questions: how to handle
failure on startup. This patch just warns and continues. I may not want my name
nodes to start up if they cant start a critical plugin
What about having configurable options, like
{{dfs.namenode.plugin.failonerror}}, and {{dfs.dataenode.plugin.failonerror}},
so that each service may decide what to do?
bq.3. Also: configuration. How do these plugins get configured. Are they
expected to sneak a look at their parent's configuration [..]
In its current form, yes, they may have a look at their parent's config (since
they get a reference to the parent in the call to
{{ServicePlugin.start(server)}}. Each plugin may also look for configuration
objects somewhere else via {{Configuration.addResource()}}.
In the context of HADOOP-3628 I suppose their parents would use something like
{{plugin.setConf(this.conf)}}.
bq.4. Finally if the plugin interface extends Closeable(), it would make it
possible for us to have utility code to handle sets of closeable items for
cleanup, usable in lots of other places too.
Good idea
> Export namenode/datanode functionality through a pluggable RPC layer
> --------------------------------------------------------------------
>
> Key: HADOOP-5257
> URL: https://issues.apache.org/jira/browse/HADOOP-5257
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Reporter: Carlos Valiente
> Priority: Minor
> Attachments: HADOOP-5257-v2.patch, HADOOP-5257-v3.patch,
> HADOOP-5257-v4.patch, HADOOP-5257-v5.patch, HADOOP-5257-v6.patch,
> HADOOP-5257-v7.patch, HADOOP-5257.patch
>
>
> Adding support for pluggable components would allow exporting DFS
> functionallity using arbitrary protocols, like Thirft or Protocol Buffers.
> I'm opening this issue on Dhruba's suggestion in HADOOP-4707.
> Plug-in implementations would extend this base class:
> {code}abstract class Plugin {
> public abstract datanodeStarted(DataNode datanode);
> public abstract datanodeStopping();
> public abstract namenodeStarted(NameNode namenode);
> public abstract namenodeStopping();
> }{code}
> Name node instances would then start the plug-ins according to a
> configuration object, and would also shut them down when the node goes down:
> {code}public class NameNode {
> // [..]
> private void initialize(Configuration conf)
> // [...]
> for (Plugin p: PluginManager.loadPlugins(conf))
> p.namenodeStarted(this);
> }
> // [..]
> public void stop() {
> if (stopRequested)
> return;
> stopRequested = true;
> for (Plugin p: plugins)
> p.namenodeStopping();
> // [..]
> }
> // [..]
> }{code}
> Data nodes would do a similar thing in {{DataNode.startDatanode()}} and
> {{DataNode.shutdown}}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
