[
https://issues.apache.org/jira/browse/STORM-789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14501695#comment-14501695
]
ASF GitHub Bot commented on STORM-789:
--------------------------------------
Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/525#issuecomment-94240470
You can temporary change multilang_test.clj to make spout -> bolt -> bolt
topology.
```clojure
(defn test-multilang
[executor file-extension]
(with-local-cluster [cluster :supervisors 4]
(let [nimbus (:nimbus cluster)
topology (thrift/mk-topology
{"1" (thrift/mk-shell-spout-spec [executor (str
"tester_spout." file-extension)] ["word"])}
{"2" (thrift/mk-shell-bolt-spec {"1" :shuffle} [executor
(str "tester_bolt." file-extension)] ["word"] :parallelism-hint 1)
"3" (thrift/mk-shell-bolt-spec {"2" :local-or-shuffle}
[executor (str "tester_bolt." file-extension)] ["word"] :parallelism-hint 1)})]
(submit-local-topology nimbus
"test"
{TOPOLOGY-WORKERS 20 TOPOLOGY-MESSAGE-TIMEOUT-SECS 3
TOPOLOGY-DEBUG true}
topology)
(Thread/sleep 10000)
(.killTopology nimbus "test")
(Thread/sleep 10000)
)))
```
Here's Spout.
```
{
"task->component":{
"20":"__acker",
"21":"__acker",
"22":"__acker",
"23":"__acker",
"1":"1",
"3":"3",
"2":"2",
"5":"__acker",
"4":"__acker",
"7":"__acker",
"6":"__acker",
"9":"__acker",
"8":"__acker",
"11":"__acker",
"10":"__acker",
"13":"__acker",
"12":"__acker",
"15":"__acker",
"14":"__acker",
"17":"__acker",
"16":"__acker",
"19":"__acker",
"18":"__acker"
},
"stream->target->grouping":{
"default":{
"2":"SHUFFLE"
}
},
"streams":[
"default"
],
"stream->outputfields":{
"default":[
"word"
]
},
"taskid":1,
"source->stream->grouping":{
},
"componentid":"1"
}
```
Bolt 1.
```
{
"task->component":{
"20":"__acker",
"21":"__acker",
"22":"__acker",
"23":"__acker",
"1":"1",
"3":"3",
"2":"2",
"5":"__acker",
"4":"__acker",
"7":"__acker",
"6":"__acker",
"9":"__acker",
"8":"__acker",
"11":"__acker",
"10":"__acker",
"13":"__acker",
"12":"__acker",
"15":"__acker",
"14":"__acker",
"17":"__acker",
"16":"__acker",
"19":"__acker",
"18":"__acker"
},
"stream->target->grouping":{
"default":{
"3":"LOCAL_OR_SHUFFLE"
}
},
"streams":[
"default"
],
"stream->outputfields":{
"default":[
"word"
]
},
"taskid":2,
"source->stream->grouping":{
"1":{
"default":"SHUFFLE"
}
},
"componentid":"2"
}
```
Bolt 2.
```
{
"task->component":{
"20":"__acker",
"21":"__acker",
"22":"__acker",
"23":"__acker",
"1":"1",
"3":"3",
"2":"2",
"5":"__acker",
"4":"__acker",
"7":"__acker",
"6":"__acker",
"9":"__acker",
"8":"__acker",
"11":"__acker",
"10":"__acker",
"13":"__acker",
"12":"__acker",
"15":"__acker",
"14":"__acker",
"17":"__acker",
"16":"__acker",
"19":"__acker",
"18":"__acker"
},
"stream->target->grouping":{
},
"streams":[
"default"
],
"stream->outputfields":{
"default":[
"word"
]
},
"taskid":3,
"source->stream->grouping":{
"2":{
"default":"LOCAL_OR_SHUFFLE"
}
},
"componentid":"3"
}
```
Looks really fine.
> Enhance topology context sent to multi-lang bolts and spouts
> ------------------------------------------------------------
>
> Key: STORM-789
> URL: https://issues.apache.org/jira/browse/STORM-789
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Dan Blanchard
> Assignee: Dan Blanchard
>
> I'm about to submit a pull request that adds a lot more contextual
> information to the "context" JSON dictionary that gets sent to multi-lang
> bolts and spouts as part of their initial handshake. These additions will
> make is so non-JVM developers have access to the sources, targets, and field
> names for their bolts and spouts.
> We will add support for this in streamparse (one of the more popular Python
> libraries for Storm) as soon as this gets merged in.
> Here are the details of what I've added:
> - componentid: the component ID for this task, so you don't have to look it
> up
> in "task->component" by "taskid"
> - streams: a list of all of the streams for this task
> - stream->outputfields: a mapping from stream names to output fields for
> that
> stream
> - stream->target->grouping: a mapping from stream names to the targets for
> this
> task to the kind of grouping they use.
> - sources->grouping: a mapping from the component IDs of the sources to
> their
> grouping.
> Note on groupings: groupings are either represented as a string (e.g.,
> "SHUFFLE") or a list of strings for field groupings.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
