sure. I will go ahead with this for now. Thanks for your suggestions. On Mon, Jul 9, 2018 at 11:10 PM, Hequn Cheng <chenghe...@gmail.com> wrote:
> Hi, > It depends on how many different users. In most cases, the performance > will be fine. I think it worth to give a try. :-) > Of course, there are ways to reduce the number of timers, for example > keyBy(userId%1024), and use a MapState to store different users for the > same group. > > On Tue, Jul 10, 2018 at 1:54 PM, anna stax <annasta...@gmail.com> wrote: > >> Thanks Hequn. I think so too, the large number of timers could be a >> problem. >> >> On Mon, Jul 9, 2018 at 10:23 PM, Hequn Cheng <chenghe...@gmail.com> >> wrote: >> >>> Hi anna, >>> >>> According to your description, I think we can use the Timer to solve >>> your problem. The TimerService deduplicates timers per key and timestamp. >>> Also, note that a large number of timers can significantly increase >>> checkpointing time. >>> >>> On Tue, Jul 10, 2018 at 11:38 AM, anna stax <annasta...@gmail.com> >>> wrote: >>> >>>> Thanks Hequn, for the links. >>>> >>>> This is my use case.. >>>> >>>> When there is no user activity for n weeks, I need to send a >>>> Notification to user. >>>> The activity stream is usually very high volume for most users. >>>> I thought it is not a good idea to use windowing for this, because of >>>> the stream volume and window size. >>>> I want to store in the state, for every user the last activity date and >>>> process them once daily. >>>> >>>> I want to make sure I am heading in the right direction. Thank you for >>>> your suggestions. >>>> >>>> -Anna >>>> >>>> On Mon, Jul 9, 2018 at 7:16 PM, Hequn Cheng <chenghe...@gmail.com> >>>> wrote: >>>> >>>>> Hi anna, >>>>> >>>>> > I need to trigger a function once every day >>>>> If you want to trigger by the function itself, you can use the >>>>> Timer[1]. Both types of timers (processing-time and event-time) are >>>>> internally maintained by the TimerService, and onTimer() method will be >>>>> called once a timer fires. >>>>> If you want to trigger the function of different >>>>> parallelism synchronously, then the broadcast state[2] may be helpful. >>>>> >>>>> Hope this helps. >>>>> Hequn >>>>> >>>>> [1] https://ci.apache.org/projects/flink/flink-docs-master/dev/s >>>>> tream/operators/process_function.html#timers >>>>> [2] https://ci.apache.org/projects/flink/flink-docs-master/d >>>>> ev/stream/state/broadcast_state.html >>>>> >>>>> On Tue, Jul 10, 2018 at 7:47 AM, anna stax <annasta...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> I need to trigger a function once every day to read the state and >>>>>> create kafka events and also remove some records from state if they are >>>>>> too >>>>>> old. >>>>>> >>>>>> Is there a way to do this? I am new to Flink, appreciate any feedback >>>>>> and suggestions. >>>>>> >>>>>> Thanks >>>>>> Anna >>>>>> >>>>> >>>>> >>>> >>> >> >
