Alec: You can create a bolt that talks to redis which provides an HLLP counter, and other counters for you:
http://redis.io/commands#hyperloglog For windowing/aggregation, you can use a key_root+hourly_timestamp, merge across the keys bounded by timestamps and then expire old keys as needed. Good luck! -Gna On Thu, Aug 21, 2014 at 2:18 PM, Nima Movafaghrad < [email protected]> wrote: > Alec, > > > > You can use something like HyperLogLog or Bloomfilters to do Unique and/or > Distinct counting. Just create a bolt that does that. > > > > Nima > > > > *From:* Sa Li [mailto:[email protected]] > *Sent:* Wednesday, August 20, 2014 2:45 PM > *To:* [email protected] > *Subject:* distinct counting > > > > Hi, all > > > > I know storm does good job on counting and other aggregate jobs, I wonder > if anyone ever did distinct counting in storm, and how would you set the > time sliding window? > > > > thanks > > > > > Alec > -- GNA PHETSARATH DIRECTOR OF TECHNOLOGY Velos Accelerating Machine Learning 440 9TH AVENUE, 11TH FLOOR NEW YORK, NY 10001 O: 917.525.2466x103 M: 917.373.7363 F: 917.525.2466x103 E: *[email protected] <[email protected]>* W: *www.velos.io <http://www.velos.io>* Twitter: @sourigna
