[
https://issues.apache.org/jira/browse/STORM-789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499962#comment-14499962
]
ASF GitHub Bot commented on STORM-789:
--------------------------------------
Github user dan-blanchard commented on the pull request:
https://github.com/apache/storm/pull/525#issuecomment-94003059
Ok, I tweaked things a bit more and I realized there were a couple little
issues with how I was building `sources->grouping`. As a result, I've had to
make that a bit more complex and turn it into `source->stream->grouping`,
because `GlobalStreamId` objects have both a component ID and a stream ID we
care about.
The new output looks like this for the bolt:
```python
{u'task->component':
{u'20': u'__acker',
u'21': u'__acker',
u'22': u'__acker',
u'1': u'1',
u'3': u'__acker',
u'2': u'2',
u'5': u'__acker',
u'4': u'__acker',
u'7': u'__acker',
u'6': u'__acker',
u'9': u'__acker',
u'8': u'__acker',
u'11': u'__acker',
u'10': u'__acker',
u'13': u'__acker',
u'12': u'__acker',
u'15': u'__acker',
u'14': u'__acker',
u'17': u'__acker',
u'16': u'__acker',
u'19': u'__acker',
u'18': u'__acker'},
u'stream->target->grouping': {},
u'streams': [u'default'],
u'stream->outputfields': {u'default': [u'word']},
u'taskid': 2,
u'source->stream->grouping': {u'1': {u'default': u'SHUFFLE'}},
u'componentid': u'2'}
```
Of course, there is one last strange part. Why is
`stream->target->grouping` empty for the bolt when `stream->outputfields` is
not? Is that to be expected for this test? Does the topology have a bolt that
emits but nothing consumes it?
> Enhance topology context sent to multi-lang bolts and spouts
> ------------------------------------------------------------
>
> Key: STORM-789
> URL: https://issues.apache.org/jira/browse/STORM-789
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Dan Blanchard
> Assignee: Dan Blanchard
>
> I'm about to submit a pull request that adds a lot more contextual
> information to the "context" JSON dictionary that gets sent to multi-lang
> bolts and spouts as part of their initial handshake. These additions will
> make is so non-JVM developers have access to the sources, targets, and field
> names for their bolts and spouts.
> We will add support for this in streamparse (one of the more popular Python
> libraries for Storm) as soon as this gets merged in.
> Here are the details of what I've added:
> - componentid: the component ID for this task, so you don't have to look it
> up
> in "task->component" by "taskid"
> - streams: a list of all of the streams for this task
> - stream->outputfields: a mapping from stream names to output fields for
> that
> stream
> - stream->target->grouping: a mapping from stream names to the targets for
> this
> task to the kind of grouping they use.
> - sources->grouping: a mapping from the component IDs of the sources to
> their
> grouping.
> Note on groupings: groupings are either represented as a string (e.g.,
> "SHUFFLE") or a list of strings for field groupings.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
