I think a message queue of some sort is mandatory for us to overcome this challenge. I looked through the architectures of several notification servers, and most of them have opted to a queue coupled with a no-sql db, to keep task status in check. In this approach we can have a set of message processors that will pick up the next available entry, process it and update the status in DB. As Chathura mentioned you will not need a separate task for re-trys in this case. However, we will need to strike a balance between the size of the message processors pool and the operations to stop the queue from growing to an unmanageable size.
Thanks and Regards, Ruwan Yatawara Associate Technical Lead, WSO2 Inc. email : [email protected] mobile : +94 77 9110413 blog : http://ruwansrants.blogspot.com/ https://500px.com/ruwan_ace https://medium.com/@ruwanyatawara www: :http://wso2.com On Thu, Mar 23, 2017 at 9:17 AM, Chathura Ekanayake <[email protected]> wrote: > Without having a separate retry task, can we have a single task running in > one one to read messages from the db and send to devices. Then the behavior > for sending incoming operation requests and sending messages after a > restart will be similar. In addition, we can avoid an in-memory store of > messages, which could cause scalability issues. > > - Chathura > > On Wed, Mar 22, 2017 at 10:41 PM, Waruna Jayaweera <[email protected]> > wrote: > >> Hi Chathura, >> >> We can not read from db , since in a clustered environment when multiple >> nodes try read from same operation table will cause duplication of push >> message sending. Anyway these bulk device request will come to given node >> at a time. So device can store them in queue for temporary for sending push >> messages as batches. If node restart we can have retry task ( Only in >> manager node) to read from database. >> >> Thanks, >> Waruna >> >> On Wed, Mar 22, 2017 at 3:58 PM, Chathura Ekanayake <[email protected]> >> wrote: >> >>> I think option 2 is better. Using a message store will also eventually >>> access a persistent storage (probably with multiple io calls), so the >>> additional db call in option 2 is not significant. In addition, option 1 >>> adds an additional hop to the message flow. >>> >>> Why do we need a concurrent queue for option 2 (as we anyway store all >>> operations in db)? Is it as a cache? >>> >>> - Chathura >>> >>> On Wed, Mar 22, 2017 at 11:52 AM, Waruna Jayaweera <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> In Device + Server Communication Push notifications used to notify >>>> devices to retrieve the device operations. This also use in policy >>>> monitoring where we send push notification to all devices during monitoring >>>> task. There are different push notification strategies as for APNS, for >>>> android - GCM/FCM and for Windows - WNS also MQTT based push notifications. >>>> >>>> *Problem* >>>> In current implementation when you set device operation to multiple >>>> devices, or during policy monitoring >>>> all push notifications are send all devices at once. This will cause >>>> sudden request burst when all devices try access the server for device >>>> operations. We also needs to think about of reliability of push >>>> notifications if those got failed. To solve this issue we have multiple >>>> solutions. >>>> >>>> *Solutions* >>>> >>>> *1) Use Message store and Message Processors* >>>> >>>> We can send each push notification message to inbuilt synapse message >>>> store via proxy. Then we can have sampling message processor to send those >>>> push notifications to given notification provider. In sampling message >>>> processor we can define no of message per given time and time interval per >>>> batch of messages. Push notification logic will be inside sequence with >>>> custom mediator. >>>> >>>> Pros; >>>> Reliability can be achieved using jms message store. >>>> minimum code >>>> >>>> Cons; >>>> IOT server have to depend on message processors. >>>> >>>> >>>> *2) Java Queue with Scheduler Task* >>>> >>>> We can have java concurrent queue to store device notifications as >>>> batch jobs. Scheduler task will be there to read jobs and process them and >>>> send notifications. Batch job object will content set of devices given >>>> batch size and scheduler task will be run based on delay interval. >>>> >>>> Pros; >>>> Do not need to rely on other feature >>>> >>>> Cons; >>>> We need to maintain push status in database so there will be additional >>>> database call in order to make reliability of notification sending. Later >>>> we can have retry logic to failed notifications based on db status. >>>> >>>> If device list is below the no of devices per batch, those commands >>>> will send immediately. >>>> IMO option 2 is better since we do not rely on any other >>>> implementation. >>>> Appreciate any suggestions. >>>> >>>> Thanks, >>>> Waruna >>>> >>>> -- >>>> Regards, >>>> >>>> Waruna Lakshitha Jayaweera >>>> Senior Software Engineer >>>> WSO2 Inc; http://wso2.com >>>> phone: +94713255198 <+94%2071%20325%205198> >>>> http://waruapz.blogspot.com/ >>>> >>>> >>> >> >> >> -- >> Regards, >> >> Waruna Lakshitha Jayaweera >> Senior Software Engineer >> WSO2 Inc; http://wso2.com >> phone: +94713255198 <+94%2071%20325%205198> >> http://waruapz.blogspot.com/ >> >> > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
