[
https://issues.apache.org/jira/browse/SAMZA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14114168#comment-14114168
]
Chris Riccomini commented on SAMZA-353:
---------------------------------------
Thanks for the feedback. After thinking about this a bit more, I think I agree
about the control channel statement. I also think that we shouldn't support
multi-subscriber partitions right now.
If we just focus on global state, then I think things might actually be fairly
easy to implement. The changes that I can think of are:
# Add config to define a state store as global. The config log config for
global stores would be the stream that's used to restore the state.
# Update the TaskStorageManager (or create some new global storage manager
class) to instantiate global stores only once per-container.
# Update TaskStorageManager (or new class) to do an initial restore for all
partitions for global stores from offset 0 to latest offset.
# Update SamzaContainer/SystemConsumers to register SystemConsumers with the
latest offset for all partitions of all global state store changelogs.
# Update RunLoop to update the global state stores rather than forwarding the
incoming changelog messages to TaskInstances.
An alternative to steps (2) and (3) is to just have the SamzaContainer define
the global state store changelog streams as bootstrap streams, and register in
step (4) with offset 0 for all changelog SSPs.
> Support assigning the same SSP to multiple tasknames
> ----------------------------------------------------
>
> Key: SAMZA-353
> URL: https://issues.apache.org/jira/browse/SAMZA-353
> Project: Samza
> Issue Type: Bug
> Components: container
> Affects Versions: 0.8.0
> Reporter: Jakob Homan
> Attachments: DESIGN-SAMZA-353-0.md, DESIGN-SAMZA-353-0.pdf
>
>
> Post SAMZA-123, it is possible to add the same SSP to multiple tasknames,
> although currently we check for this and error out if this is done. We
> should think through the implications of having the same SSP appear in
> multiple tasknames and support this if it makes sense.
> This could be used as a broadcast stream that's either added by Samza itself
> to each taskname, or individual groupers could do this as makes sense. Right
> now the container maintains a map of SSP to TaskInstance and delivers the ssp
> to that task instance. With this change, we'd need to change the map to SSP
> to Set[TaskInstance] and deliver the message to each TI in the set.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
