[
https://issues.apache.org/jira/browse/STORM-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rick Kellogg updated STORM-36:
------------------------------
Component/s: storm-core
> partitionPersist that can take as input multiple streams with differing
> schemas
> -------------------------------------------------------------------------------
>
> Key: STORM-36
> URL: https://issues.apache.org/jira/browse/STORM-36
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Reporter: James Xu
> Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/369
> Each stream may do different actions to the State object.
> -----------------------------------------------------------------------------------------------------
> quintona: I think this would solve an issue I am having, let me explain the
> use case. I essentially have a single spout emitting links. The content of
> the link is then downloaded and analyzed. Various parallel logical streams
> are then derived, some relatively static compared to the others. What I mean
> by that is some state is log living, well beyond the batch, like the number
> of links. This is state increments with each tuple within and across batches.
> At the same time the actual main stream is deriving measures at the batch
> level, and these 2 figures need to be combined into a single expression
> later. The approach I was using was to persist the long running count using
> the persistentAggregate, then I was intending to use a statequery to derive a
> stream off that state and merge that into main stream. I could therefore have
> a single function receiving a batch level count, and a running total across
> time which it needs in order to do its calculation.
> The only other approach I can think of to achieve this (given that I can't
> merge the streams suggested above, and I have no means of joining), is to
> have multiple streams effect a single state as you suggest here, or a
> periodic DRPC approach.
> If this the sort of thing you were trying to solve here?
> Here is potentially one approach? https://gist.github.com/quintona/5558787
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
