Parth Brahmbhatt created STORM-1381:
---------------------------------------
Summary: Client side topology submission hook.
Key: STORM-1381
URL: https://issues.apache.org/jira/browse/STORM-1381
Project: Apache Storm
Issue Type: New Feature
Components: storm-core
Affects Versions: 0.11.0
Reporter: Parth Brahmbhatt
Assignee: Parth Brahmbhatt
Priority: Trivial
Fix For: 0.11.0
A client side hook is suppose to be invoked when a user submits the topology
using TopologySubmitter. We already have nimbus side hook for all the topology
actions however those are good if users don't want to actually inspect the
topology being submitted or the classes that makes up the topology (spouts and
bolts) as on nimbus side these classes are not available in class path.
As a concrete example, in hortonworks we wanted to integrate storm with atlas
to provide complete lineage of data even when it passes through a storm
topology. Atlas needed to actually look inside the topology components (i.e.
kafka spout to figure out what topic the data is being pulled from, or hbase
bolt to figure out which cluster and what table data is being pushed into.) to
give a meaningful lineage. We originally proposed that they use the server side
hook but with that they had to download the user uploaded jar and add it to the
class path dynamically or spin a new jvn whose output will then be read by the
atlas integration hook.
The client side hook is suppose to make it easy when the topology itself needs
to be examined. We are using this in our internal repo for atlas integration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
