[
https://issues.apache.org/jira/browse/STORM-789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499912#comment-14499912
]
ASF GitHub Bot commented on STORM-789:
--------------------------------------
Github user dan-blanchard commented on the pull request:
https://github.com/apache/storm/pull/525#issuecomment-93998284
I modified my local copy of storm.py to make it log the contexts its
received and I noticed that `TesterSpout` gets:
```python
{u'task->component':
{u'20': u'__acker',
u'21': u'__acker',
u'22': u'__acker',
u'1': u'1',
u'3': u'__acker',
u'2': u'2',
u'5': u'__acker',
u'4': u'__acker',
u'7': u'__acker',
u'6': u'__acker',
u'9': u'__acker',
u'8': u'__acker',
u'11': u'__acker',
u'10': u'__acker',
u'13': u'__acker',
u'12': u'__acker',
u'15': u'__acker',
u'14': u'__acker',
u'17': u'__acker',
u'16': u'__acker',
u'19': u'__acker',
u'18': u'__acker'},
u'sources->grouping': {},
u'stream->target->grouping': {u'default': {u'2': u'SHUFFLE'}},
u'streams': [u'default'],
u'stream->outputfields': {u'default': [u'word']},
u'taskid': 1,
u'componentid': u'1'}
```
and the corresponding `TesterBolt` gets:
```python
{u'task->component':
{u'20': u'__acker',
u'21': u'__acker',
u'22': u'__acker',
u'1': u'1',
u'3': u'__acker',
u'2': u'2',
u'5': u'__acker',
u'4': u'__acker',
u'7': u'__acker',
u'6': u'__acker',
u'9': u'__acker',
u'8': u'__acker',
u'11': u'__acker',
u'10': u'__acker',
u'13': u'__acker',
u'12': u'__acker',
u'15': u'__acker',
u'14': u'__acker',
u'17': u'__acker',
u'16': u'__acker',
u'19': u'__acker',
u'18': u'__acker'},
u'sources->grouping': {},
u'stream->target->grouping': {},
u'streams': [u'default'],
u'stream->outputfields': {u'default': [u'word']},
u'taskid': 2,
u'componentid': u'2'}
```
`stream->target->grouping` seems to be working fine, but for some reason
`TesterSpout` isn't showing up in `sources->grouping` (as I would expect it to,
since its the source for `TesterBolt`). Let me fix that.
> Enhance topology context sent to multi-lang bolts and spouts
> ------------------------------------------------------------
>
> Key: STORM-789
> URL: https://issues.apache.org/jira/browse/STORM-789
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Dan Blanchard
> Assignee: Dan Blanchard
>
> I'm about to submit a pull request that adds a lot more contextual
> information to the "context" JSON dictionary that gets sent to multi-lang
> bolts and spouts as part of their initial handshake. These additions will
> make is so non-JVM developers have access to the sources, targets, and field
> names for their bolts and spouts.
> We will add support for this in streamparse (one of the more popular Python
> libraries for Storm) as soon as this gets merged in.
> Here are the details of what I've added:
> - componentid: the component ID for this task, so you don't have to look it
> up
> in "task->component" by "taskid"
> - streams: a list of all of the streams for this task
> - stream->outputfields: a mapping from stream names to output fields for
> that
> stream
> - stream->target->grouping: a mapping from stream names to the targets for
> this
> task to the kind of grouping they use.
> - sources->grouping: a mapping from the component IDs of the sources to
> their
> grouping.
> Note on groupings: groupings are either represented as a string (e.g.,
> "SHUFFLE") or a list of strings for field groupings.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
