Thanks Deepak for the reply .
Yes , here as well Storm will be acting as a subscriber for Incoming Event
and we plan to use this for aggregation (Map and Reduce) in real time. The
volume of events can be huge here .
I can u a rough idea (sorry if i may sound opaque at some points as we are
just trying to connect dots at this moment).
So , the idea here is , we are trying to build a classic Pub/Sub model
that we will have a message Bus (Kafka or RabbitMq) where we will get some
events .
To elaborate an event , there will be various Topics (which will be some
String Value) and we will get events for topics which will nothing but some
metadata for that evebt . Now , we will Figure out what all mobile users
have subscribed to this particular event and append it to event msg .
To give a very rough example (since we are also not sure what will an
actual event look like as at this point we are just finding ways to
define/solve this problem) :
Say There is Topic say TomCruise and we get an event in json (containing
metadata for his upcoming movie release) . This event , we will receive in
our Message Bus (say kafka or rabbit Mq)
Event -> {TomCruise , {EventBody} } .
We plan to feed this stream to Apache Storm Spout where in it calls 2
bolts (one for Map and One for Reduce)
In the Bolt1 ,we may be doing a look up as to which all mobile users have
subscribed to this Topic's Event and their type and then Change the above
to
{ {Event1},{MobileUser1} ,{iOS} } .
{ {Event2},{MobileUser2} ,{GCM} } .
{ {Event1} , {MobileUser3},{iOs}}
{ {Event2} , {MobileUser4}.{GCM}}
..and so on
iOs,GCM are delivery channels (GCM stands for Google Cloud Messaging) so
that we can push to Android devices.
And in Bolt2 , we do aggregation that which Event needs to be delivered to
which set of users
{
Messages {
message { 'iOS', 'Event1', listOfMobileUsers {'MobileUser1',
'MobileUser3.....'} },
message { 'GCM', 'Even2', listOfMobileUsers {'MobileUser2',
'MobileUser4.....'} },
and So on
}
>From here we will again push this to a notification Message Bus and write
plugins which will be responsible for delivering them to end user .
Now,since we will be having potentially millions of subscribers . We want
to build a platform which can handle this scale reliably . Hence , thinking
of evaluating Storm for this .
Hope I could describe the use case here .
Thanks
Ankur
On Mon, Aug 31, 2015 at 11:04 PM, Deepak Sharma <[email protected]>
wrote:
> Hi Ankur
> I have not seen this kind of setup earlier.But i would say that storm is
> basically the subscriber in majority of implementations that i have seen or
> worked upon.
> So here you can use storm to process the events/notifications and then
> push it from storm to different topics where the mobile users are
> subscribed to.
>
> HTH
> --Deepak
>
> On Mon, Aug 31, 2015 at 10:59 PM, Ankur Garg <[email protected]> wrote:
>
>> Hi ,
>>
>> We in our organization are building a system to send real time
>> notification (Pub-Sub) to Mobile Users(Android and IOS) .
>>
>> Broadly the use case is that mobile users subscribe to some topics and
>> whenever there is/are events related to this topic , we need to send those
>> users a Notification for that message .
>>
>> The estimate is we are going to have a large no of users over a period of
>> time (may be millions) to get these notifications .
>>
>> Since Storm is used for real -time processing of data, I am planning to
>> explore this whether it fits our requirement .
>>
>> Does storm fits the bill here ? Is it being used somewhere to do this ?
>>
>> Looking for some pointers .
>>
>> Thanks
>> Ankur
>>
>
>
>
> --
> Thanks
> Deepak
> www.bigdatabig.com
> www.keosha.net
>
