Re: updateStateByKey and invFunction

2015-02-24 Thread Ashish Sharma
But how will I specify my state there? On Tue, Feb 24, 2015 at 12:50 AM Arush Kharbanda wrote: > You can use a reduceByKeyAndWindow with your specific time window. You can > specify the inverse function in reduceByKeyAndWindow. > > On Tue, Feb 24, 2015 at 1:36 PM, Ashish Sharma > wrote: > >> So

Re: updateStateByKey and invFunction

2015-02-24 Thread Arush Kharbanda
You can use a reduceByKeyAndWindow with your specific time window. You can specify the inverse function in reduceByKeyAndWindow. On Tue, Feb 24, 2015 at 1:36 PM, Ashish Sharma wrote: > So say I want to calculate top K users visiting a page in the past 2 hours > updated every 5 mins. > > so here

updateStateByKey and invFunction

2015-02-24 Thread Ashish Sharma
So say I want to calculate top K users visiting a page in the past 2 hours updated every 5 mins. so here I want to maintain something like this Page_01 => {user_01:32, user_02:3, user_03:7...} ... Basically a count of number of times a user visited a page. Here my key is page name/id and state i