[
https://issues.apache.org/jira/browse/METRON-638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15768444#comment-15768444
]
Ryan Merriman commented on METRON-638:
--------------------------------------
Here are my thoughts on an initial design. I have attached stubs of the
classes I think we'll need.
The ZkConfigurationManager class will look almost exactly like the old
ConfiguredBolt class. ZkConfigurationManager.start will be almost identical
to ConfiguredBolt.prepare. There will also be an extra method that exposes the
zookeeper root parameter. This class is essentially just a wrapper around
Curator. The subclasses of this class would provide the ZK root, handle
deserialization to the appropriate types and so on.
The ParserConfigurationManager class would replace ConfiguredParserBolt
(likewise for other topologies, ConfiguredEnrichmentBolt, etc). The
ConfiguredParserBolt.loadConfig method would move to
ParserConfigurationManager.start (because that's when it happens, no need for
an abstract method in the parent class I think). Since the bolts no longer
inherit from ConfiguredBolt and don't implement a reloadCallback method, we
need a way to pass in callback functions.
ParserConfigurationManager.addCallback should solve that.
One design change I propose is to move global config into it's own
GlobalConfigurationManager class instead of being combined with other configs.
I believe this will simplify the logic in the various updateConfig methods
because we won't have to check the path and it will make everything clearer and
easier to understand. We should also discuss whether we want to leverage a
cache (if we think it will provide some benefit) or stick with what we have
now, which is storing deserialized objects in a ConcurrentMap.
We will of course have to adjust all the child bolts to use this new pattern.
For example the ParserBolt would change as follows:
public class ParserBolt extends BaseRichBolt { // does not extend
ConfiguredBolt anymore
private GlobalConfigurationManager globalConfigurationManager;
private ParserConfigurationManager parserConfigurationManager;
public ParserBolt( String zookeeperUrl
, String sensorType
, MessageParser<JSONObject> parser
, WriterHandler writer
)
{
//super(zookeeperUrl, sensorType); no longer needed
this.globalConfigurationManager = new
GlobalConfigurationManager().withZookeeperUrl(zookeeperUrl);
this.parserConfigurationManager = new
ParserConfigurationManager().withZookeeperUrl(zookeeperUrl);
this.writer = writer;
this.parser = parser;
}
public void prepare(Map stormConf, TopologyContext context, OutputCollector
collector) {
this.globalConfigurationManager.start();
this.parserConfigurationManager.start();
//do other stuff
}
//global config example
protected void initializeStellar() {
this.stellarContext = new Context.Builder()
.with(Context.Capabilities.ZOOKEEPER_CLIENT, ()
-> client)
.with(Context.Capabilities.GLOBAL_CONFIG, () ->
this.globalConfigurationManager.get())
.build();
StellarFunctions.initialize(stellarContext);
}
//parser config example
@SuppressWarnings("unchecked")
@Override
public void execute(Tuple tuple) {
byte[] originalMessage = tuple.getBinary(0);
SensorParserConfig sensorParserConfig =
this.parserConfigurationManager.get().getSensorParserConfig(sensorType);
//do other stuff
}
@Override
public void cleanup() {
this.globalConfigurationManager.stop();
this.parserConfigurationManager.stop();
}
}
This should get us started. Let me know what you all think about this
approach, what is missing or what should be changed.
Ryan
> Refactor ConfiguredBolt
> -----------------------
>
> Key: METRON-638
> URL: https://issues.apache.org/jira/browse/METRON-638
> Project: Metron
> Issue Type: Bug
> Reporter: Ryan Merriman
> Assignee: Ryan Merriman
> Attachments: GlobalConfigurationManager.java,
> ParserConfigurationManager.java, ZkConfigurationManager.java
>
>
> Currently ConfiguredBolt extends BaseRichBolt and is included in other bolts
> through inheritance. This limits it's use to only a subset of Storm Bolts:
> those that also extend BaseRichBolt. Bolts that must extends a different
> Base Bolt (BaseWindowBolt for example) cannot use it. This component could
> be useful outside of Storm as well.
> We need to refactor the ConfiguredBolt into a separate class that can be
> included in any other class through composition rather than inheritance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
