[
https://issues.apache.org/jira/browse/SAMZA-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16811149#comment-16811149
]
Sanil Jain commented on SAMZA-2116:
-----------------------------------
Update:
We will not make sink non-terminal, since we today have no way to control the
sink operator output. Since sink operator gives flexibility to for users to use
MessageCollector and TaskCoordinator, this gives users to change the input
message stream, write to output streams. Giving users the way to control the
output of sync operator does not seem to be a good solution
> Make sendTo operators non-terminal
> ----------------------------------
>
> Key: SAMZA-2116
> URL: https://issues.apache.org/jira/browse/SAMZA-2116
> Project: Samza
> Issue Type: Improvement
> Reporter: Prateek Maheshwari
> Assignee: Sanil Jain
> Priority: Major
>
> sendTo(stream), sendTo(table) and sink are currently terminal operators in
> the High Level API. They don't return a MessageStream, and hence do not allow
> chaining of further operators.
> Consider the following use case:
> Consume a message -> send an event denoting processing start -> do some
> processing -> send an event denoting processing end.
> Currently there's no good way to do this, since:
> # HL API does not allow chaining further operations on sendTo
> # HL API does not guarantee order of execution across multiple operators
> chained on the same operator.
> The only way to do this today will be to use a sink function, which is not
> great.
> One consideration when making sendTo(stream) et al. non-terminal: unlike
> partitionBy, which returns data in the re-keyed MessageStream partition,
> sendTo will return the data in the original message stream partition. This
> will need to be clearly called out.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
