[
https://issues.apache.org/jira/browse/FLINK-16319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051433#comment-17051433
]
Dan Pettersson edited comment on FLINK-16319 at 3/4/20, 5:18 PM:
-----------------------------------------------------------------
Hi again and I hope you don't mind that I'm reaching out on this Jira ticket.
For the publishing of event we it would be nice with a persistent attribute
that would work like a FIFO queue. The PersistentTable could also be a good
option (As one for example can hold the number retries that has been done for
an address if the publish has failed that function etc)
I've googled around but I haven't found any answers if the PersistentTable i.e.
the MapState guarantees the insertion order. Does anyone of you know?
Thanks,
/Dan
was (Author: danp11):
Hi again and I hope you don't mind that I'm reaching out on this Jira ticket.
For the publishing of event we it would be nice with a persistent attribute
that would work like a FIFO queue. The PersistentTable could also be a good
option (As one for example can hold the number retries that has been done for
an address if the publish has failed that option etc)
I've googled around but I haven't found any answers if the PersistentTable i.e.
the MapState guarantees the insertion order. Does anyone of you know?
Thanks,
/Dan
> Pubsub-/Broadcast implementation
> --------------------------------
>
> Key: FLINK-16319
> URL: https://issues.apache.org/jira/browse/FLINK-16319
> Project: Flink
> Issue Type: New Feature
> Components: Stateful Functions
> Reporter: Dan Pettersson
> Priority: Major
>
> Hi everyone,
>
> I have a use case where the id of the functions are brokerId + instrumentId
> that receives trades and orders. The instrument has state changes (Open,
> halted, closed etc) that almost all the functions are interested in. Some
> functions only wants for example the Close message whereas other functions
> wants all state changes for the specific instrument.
>
> I've built a statefun pubsub module that exposes two interfaces, Subscriber
> and Publisher, with these two methods:
>
> default void subscribe(Context context, Subscription... subscriptions)
>
> default void publish(Context context, PublishMessage publishMessage)
>
> Behind the interfaces is a hidden StatefulPubSubFunction that keeps track of
> which partition the subscriber is located in and to which topic it listens to.
>
> Code is located under
> [https://github.com/danp11/flink-statefun/tree/master/statefun-pubsub] if
> anyone is interested.
>
> This code is a "classic pub sub" pattern and I think that this kind of
> functionality would be a great addition to Stateful functions. I create this
> Jira to see if there is an interest to discuss how a optimal
> pubsub-/broadcast solution would look like in SF? Igal has previously
> mentioned that Broadcast could be a good fit for this kind of flow.
> At the moment I don't know the internals of SF and-/or Flink good enough to
> come up with a proposal myself unfortunately.
>
> I know you are very busy at the moment (Its impressive how much you have
> produced only the last couple of weeks!:-) but if someone, on a high level,
> has any ideas on where and how a pub sub pattern could be implemented I'd
> really appreciate it. In the future I hope we can come up with a proposal
> together as I need your help here. If you think that a pubsub-/broadcast
> solution would make SF better that is :-)
>
> Hope to hear your thoughts on this!
>
> Thanks,
>
> /Dan
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
